-- Logs begin at Sun 2022-09-11 10:07:48 UTC, end at Sun 2022-09-11 11:12:29 UTC. -- Sep 11 10:32:11.790470 dv-cinder-856460-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 11 10:32:11.799359 dv-cinder-856460-3-iscsi devstack@keystone.service[90618]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 11 10:32:11.801327 dv-cinder-856460-3-iscsi devstack@keystone.service[90618]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sun Sep 11 10:32:11 2022] *** Sep 11 10:32:11.801327 dv-cinder-856460-3-iscsi devstack@keystone.service[90618]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 11 10:32:11.801327 dv-cinder-856460-3-iscsi devstack@keystone.service[90618]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 11 10:32:11.801327 dv-cinder-856460-3-iscsi devstack@keystone.service[90618]: nodename: dv-cinder-856460-3-iscsi Sep 11 10:32:11.801327 dv-cinder-856460-3-iscsi devstack@keystone.service[90618]: machine: x86_64 Sep 11 10:32:11.801327 dv-cinder-856460-3-iscsi devstack@keystone.service[90618]: clock source: unix Sep 11 10:32:11.801327 dv-cinder-856460-3-iscsi devstack@keystone.service[90618]: pcre jit disabled Sep 11 10:32:11.801327 dv-cinder-856460-3-iscsi devstack@keystone.service[90618]: detected number of CPU cores: 4 Sep 11 10:32:11.801327 dv-cinder-856460-3-iscsi devstack@keystone.service[90618]: current working directory: / Sep 11 10:32:11.801327 dv-cinder-856460-3-iscsi devstack@keystone.service[90618]: detected binary path: /usr/bin/uwsgi-core Sep 11 10:32:11.801852 dv-cinder-856460-3-iscsi devstack@keystone.service[90618]: your processes number limit is 47680 Sep 11 10:32:11.801852 dv-cinder-856460-3-iscsi devstack@keystone.service[90618]: your memory page size is 4096 bytes Sep 11 10:32:11.801852 dv-cinder-856460-3-iscsi devstack@keystone.service[90618]: detected max file descriptor number: 2048 Sep 11 10:32:11.801852 dv-cinder-856460-3-iscsi devstack@keystone.service[90618]: lock engine: pthread robust mutexes Sep 11 10:32:11.801852 dv-cinder-856460-3-iscsi devstack@keystone.service[90618]: thunder lock: enabled Sep 11 10:32:11.801852 dv-cinder-856460-3-iscsi devstack@keystone.service[90618]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 11 10:32:11.801852 dv-cinder-856460-3-iscsi devstack@keystone.service[90618]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 11 10:32:11.843213 dv-cinder-856460-3-iscsi devstack@keystone.service[90618]: Python main interpreter initialized at 0x55befe8079c0 Sep 11 10:32:11.843213 dv-cinder-856460-3-iscsi devstack@keystone.service[90618]: python threads support enabled Sep 11 10:32:11.843213 dv-cinder-856460-3-iscsi devstack@keystone.service[90618]: your server socket listen backlog is limited to 100 connections Sep 11 10:32:11.843213 dv-cinder-856460-3-iscsi devstack@keystone.service[90618]: your mercy for graceful operations on workers is 90 seconds Sep 11 10:32:11.843617 dv-cinder-856460-3-iscsi devstack@keystone.service[90618]: mapped 403077 bytes (393 KB) for 2 cores Sep 11 10:32:11.843717 dv-cinder-856460-3-iscsi devstack@keystone.service[90618]: *** Operational MODE: preforking *** Sep 11 10:32:11.843943 dv-cinder-856460-3-iscsi devstack@keystone.service[90618]: *** uWSGI is running in multiple interpreter mode *** Sep 11 10:32:11.843943 dv-cinder-856460-3-iscsi devstack@keystone.service[90618]: spawned uWSGI master process (pid: 90618) Sep 11 10:32:11.844149 dv-cinder-856460-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 11 10:32:11.846932 dv-cinder-856460-3-iscsi devstack@keystone.service[90618]: spawned uWSGI worker 1 (pid: 90653, cores: 1) Sep 11 10:32:11.846932 dv-cinder-856460-3-iscsi devstack@keystone.service[90618]: spawned uWSGI worker 2 (pid: 90654, cores: 1) Sep 11 10:32:11.846932 dv-cinder-856460-3-iscsi devstack@keystone.service[90618]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 11 10:32:16.186885 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 11 10:32:16.186885 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: from cryptography.utils import int_from_bytes Sep 11 10:32:16.187689 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 11 10:32:16.187689 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: from cryptography.utils import int_from_bytes Sep 11 10:32:16.187802 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 11 10:32:16.187802 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: from cryptography.utils import int_from_bytes Sep 11 10:32:16.188205 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 11 10:32:16.188205 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: from cryptography.utils import int_from_bytes Sep 11 10:32:16.238520 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 11 10:32:16.238697 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 11 10:32:16.238801 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 11 10:32:16.238893 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 11 10:32:16.238967 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] command line args: [] {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 11 10:32:16.239044 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] command line args: [] {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 11 10:32:16.239156 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 11 10:32:16.239282 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 11 10:32:16.239356 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] ================================================================================ {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 11 10:32:16.239449 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] ================================================================================ {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 11 10:32:16.239602 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] admin_token = **** {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.239696 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] admin_token = **** {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.239851 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] config_dir = [] {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.239937 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] config_dir = [] {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.240040 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.240124 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.240203 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] config_source = [] {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.240273 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] config_source = [] {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.240386 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] debug = True {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.240471 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] debug = True {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.240583 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.240832 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.240932 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.240999 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.241136 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] insecure_debug = False {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.241221 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] insecure_debug = False {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.241350 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.241435 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.241598 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.241682 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.241827 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] list_limit = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.241905 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] list_limit = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.242013 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] log_config_append = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.242093 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] log_config_append = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.242164 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.242229 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.242342 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] log_dir = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.242415 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] log_dir = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.242515 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] log_file = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.242593 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] log_file = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.242743 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.242822 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.242987 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.243079 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.243236 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.243315 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.243507 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.243598 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.243668 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.243741 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.243849 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.244006 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.244125 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.244290 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.244360 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.244601 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.244699 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.244836 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.244922 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.245072 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.245156 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.245311 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.245433 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.245561 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.245647 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.245809 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] notification_format = cadf {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.245928 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.246053 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] notification_format = cadf {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.246140 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.246333 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.246567 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.15/identity {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.246754 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] publish_errors = False {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.246829 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.15/identity {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.246982 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] publish_errors = False {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.247066 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.247216 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.247294 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.247411 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.247525 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.247636 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.247721 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.247883 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.247971 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.248117 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] standard_threads = False {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.248193 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.248349 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] standard_threads = False {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.248426 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] strict_password_check = False {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.248580 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] strict_password_check = False {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.248660 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.248758 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] use_eventlog = False {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.248836 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.248937 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] use_journal = False {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.249015 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] use_eventlog = False {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.249138 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] use_json = False {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.249219 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] use_journal = False {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.249297 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] use_stderr = False {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.249369 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] use_json = False {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.249446 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] use_syslog = False {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.249523 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] use_stderr = False {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.249628 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] use_syslog = False {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.249707 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] watch_log_file = False {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.249859 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] watch_log_file = False {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 10:32:16.249937 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.250111 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.250195 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.250348 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.250427 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.250599 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.250675 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.250835 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.250911 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.251079 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.251157 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.251312 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.251389 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.251586 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.251666 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.251824 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.251905 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.252073 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.252150 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.252319 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.252395 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.252553 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.252632 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.252784 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.252862 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.253028 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.253106 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.253264 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.253344 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.253511 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.253597 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.253744 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.253822 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.253980 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.254064 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.254233 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.254311 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.254477 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.254630 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.254706 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.254778 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.254938 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.255015 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.255189 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.255282 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.255469 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.255551 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.255724 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.255803 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.255960 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.256046 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.256189 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.256266 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.256421 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.256499 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.256651 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.256737 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.256885 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.256964 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.257133 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.257212 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.257366 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.257451 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.257598 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.257677 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.257825 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.257901 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.258072 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.258158 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.258311 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.258394 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.258544 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.258622 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.258783 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.258872 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.259010 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.259087 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.259245 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.259321 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.259497 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.259610 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.259844 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.259978 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.260283 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.260367 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.260528 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.260676 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.260752 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.260823 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.260991 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.261070 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.261244 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.261332 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.261478 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.261562 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.261715 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.261827 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.261945 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.262059 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.262182 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.262293 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.262418 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.262529 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.262653 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.262747 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.262886 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.262963 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.263123 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.263202 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.263356 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.263464 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.263615 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.263697 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.263852 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.263941 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.264083 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.264167 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.264311 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.264393 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.264562 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.264640 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.264798 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.264878 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.265031 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.265124 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.265261 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.265431 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.265521 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.265595 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.265749 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.265829 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.265978 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.266105 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.266213 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.266297 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.266447 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.266525 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.266715 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.266796 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.266953 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.267033 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.267200 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.267287 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.267476 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.267561 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.267696 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.267773 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.267932 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.268012 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.268162 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.268239 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] cache.enabled = True {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.268410 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] cache.enabled = True {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.268495 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.268647 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.268727 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.268881 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.268958 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.269122 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.269199 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.269356 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.269436 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.269602 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.269679 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.269838 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.269916 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.270075 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.270154 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.270307 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.270385 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.270542 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.270620 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.270775 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.270860 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.271037 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.271117 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.271279 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.271357 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.271545 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.271625 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.271785 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.271863 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.272021 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.272100 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.272250 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.272325 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.272481 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.272605 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.272713 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.272799 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.272942 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.273030 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.273170 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.273249 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.273399 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.273477 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.273627 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.273712 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.273857 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.273935 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.274085 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.274163 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.274323 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.274411 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.274555 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.274633 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.274800 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.274877 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.275033 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.275122 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.275914 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.275997 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.276212 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.276474 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.276594 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] auth.external = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.276678 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] auth.mapped = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.276752 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.276903 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] auth.external = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.276994 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.277137 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] auth.mapped = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.277223 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.277402 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] auth.password = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.277481 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.277637 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] auth.token = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.277718 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.277874 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] auth.password = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.277979 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.278108 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] auth.token = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.278221 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] catalog.caching = True {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.278347 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.278461 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.278581 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.278829 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] catalog.caching = True {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.278902 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.279078 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.279156 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.279309 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.279378 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.279575 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] credential.caching = True {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.279646 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.279822 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.279901 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] credential.driver = sql {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.280055 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.280125 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.280365 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] credential.caching = True {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.280467 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.280533 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] credential.driver = sql {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.280601 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.280778 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.280854 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.281017 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.281102 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.281260 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.281347 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.281501 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.281576 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.281734 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.281803 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.281980 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.282057 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.282236 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.282305 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.282470 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.282538 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.282723 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.282800 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.282979 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.283053 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.283240 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.283328 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.283502 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] federation.caching = True {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.283602 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.283744 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.283832 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.283995 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] federation.driver = sql {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.284083 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.284226 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] federation.caching = True {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.284320 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.284466 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.284554 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.284711 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.284796 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] federation.driver = sql {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.284951 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.285030 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.285176 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.285254 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.285423 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.285501 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.285662 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.285741 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.285905 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.285983 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.286144 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.286228 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.286377 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] identity.caching = True {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.286457 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.286620 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.286698 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.286868 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.286947 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.287094 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] identity.caching = True {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.287169 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.287329 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.287407 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.287598 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.287676 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] identity.driver = sql {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.287830 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.287909 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.288063 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.288141 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.288310 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.288387 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] identity.driver = sql {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.288538 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.288616 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.288771 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.288845 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.289002 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.289079 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.289236 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.289316 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.289466 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.289544 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.289696 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.289775 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.289926 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.290006 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.290166 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.290252 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.290411 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.290490 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.290658 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.290737 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.291454 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.291546 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.291713 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.291953 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.292088 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.292192 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.292272 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.292425 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.292503 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.292664 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.292751 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.292907 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.292985 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.293150 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.293228 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.293381 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.293470 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.293620 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.293697 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.293861 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.293938 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.294103 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.294190 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.294337 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.294414 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.294574 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.294662 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.294823 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.294902 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.295054 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.295132 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.295291 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.295368 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.295554 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] ldap.password = **** {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.295635 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.295800 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.295879 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.296031 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.296112 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.296273 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.296354 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.296501 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] ldap.password = **** {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.296578 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.296737 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.296857 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.296973 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.297113 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.297213 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.297367 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.297445 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.297604 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.297792 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.297871 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.298046 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.298125 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.298300 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.298378 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.298535 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.298615 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.298769 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.298848 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.299007 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.299085 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.299242 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] ldap.user = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.299322 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.299509 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.299590 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.299748 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.299826 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.299987 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.300067 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.300219 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] ldap.user = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.300297 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.300464 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.300549 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.300713 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.300791 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.300948 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.301026 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.301179 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.301256 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.301418 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.301498 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.301654 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.301732 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.301892 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.301970 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.302119 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.302203 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.302355 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.302435 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.302592 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.302670 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.302829 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.302908 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.303059 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.303177 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.303308 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.303386 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.303573 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.303655 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.303818 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.303896 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.304055 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.304132 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.304294 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.304373 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.304521 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.304598 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.304756 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.304833 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.304990 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.305068 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.305225 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.305304 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.305461 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.305555 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.306071 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.306202 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] policy.driver = sql {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.306345 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.306478 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.306478 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.306695 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.306853 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.306996 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.307101 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] receipt.caching = True {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.307198 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] policy.driver = sql {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.307295 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.307295 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.307559 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.307662 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.307759 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.307856 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.307953 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.308050 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.308162 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] receipt.caching = True {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.309964 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] resource.caching = True {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.310244 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.310368 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.310486 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.310617 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] resource.driver = sql {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.310736 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.310852 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.310969 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.311085 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.311200 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.311323 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.311462 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] resource.caching = True {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.311594 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] revoke.caching = True {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.311712 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.311827 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.311942 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] resource.driver = sql {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.312068 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.312184 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.312298 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] role.cache_time = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.312413 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.312527 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] role.caching = True {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.312717 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.312835 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] role.driver = sql {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.312951 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] revoke.caching = True {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.313066 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] role.list_limit = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.313181 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.313296 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.313419 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.313535 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.313651 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] role.cache_time = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.313765 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.313880 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] role.caching = True {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.313995 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.314118 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] role.driver = sql {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.314233 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.314348 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] role.list_limit = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.314462 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.314577 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.314690 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.314814 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.314930 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.315044 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.315159 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.315274 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.315388 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.315564 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.315682 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.315797 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.315912 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.316025 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.316139 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.316263 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.316263 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.316515 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.316630 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.316743 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.316857 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.316980 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.317096 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.317210 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.317326 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.317440 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.317440 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.317661 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.317777 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.317890 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.318004 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.318117 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.318239 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.318357 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.318471 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.318584 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.318702 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.318816 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.318938 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.319054 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.319157 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.319254 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.319371 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.319500 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.319628 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.319745 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.319859 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.319972 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.320086 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.320200 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.320322 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.320438 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.320551 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] token.cache_time = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.320665 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.320779 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] token.caching = True {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.320892 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.321013 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.321128 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.321279 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] token.provider = fernet {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.321397 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.321512 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.321626 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.321748 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.321864 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] token.cache_time = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.322007 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.322153 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] token.caching = True {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.322269 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.322382 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.322506 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.322621 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] token.provider = fernet {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.322735 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.322849 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.322962 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] trust.driver = sql {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.323076 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.323198 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.323313 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.323452 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.323572 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.323686 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.323800 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.323926 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.324043 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.324043 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] trust.driver = sql {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.324256 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.324256 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.324477 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.324601 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.324725 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.324841 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.324956 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.325070 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] database.connection = **** {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.325185 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.325299 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.325299 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.325523 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.325523 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.325747 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.325870 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.325986 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.325986 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.326199 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.326321 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.326480 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] database.connection = **** {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.326598 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.326714 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.326829 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.326944 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.327067 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.327183 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.327297 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.327412 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.327537 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.327651 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.327778 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.327896 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.328010 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.328124 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.328238 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.328359 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.328488 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.328608 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.328724 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.328839 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.328954 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.329068 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.329190 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.329306 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.329421 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=90654) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 11 10:32:16.329536 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.329536 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.329536 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.329536 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.329536 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.329536 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 10:32:16.329536 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=90653) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 11 10:32:16.357761 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=90654) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 10:32:16.357987 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=90654) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 10:32:16.358143 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=90654) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 10:32:16.358545 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=90653) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 10:32:16.358777 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=90653) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 10:32:16.358998 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=90653) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 10:32:16.366445 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=90654) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 10:32:16.367230 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=90653) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 10:32:16.394422 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=90654) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 10:32:16.394653 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=90654) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 10:32:16.395174 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=90653) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 10:32:16.395406 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=90653) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 10:32:16.448882 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=90654) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 10:32:16.449273 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=90653) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 10:32:16.497403 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=90654) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 10:32:16.497638 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=90654) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 10:32:16.498307 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=90653) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 10:32:16.498542 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=90653) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 10:32:16.653637 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=90654) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 10:32:16.653866 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=90654) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 10:32:16.654090 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=90654) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 10:32:16.654309 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=90654) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 10:32:16.654484 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=90653) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 10:32:16.654571 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=90654) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 10:32:16.654716 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=90653) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 10:32:16.654799 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=90654) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 10:32:16.654937 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=90653) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 10:32:16.655155 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=90653) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 10:32:16.655370 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=90653) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 10:32:16.655610 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=90653) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 10:32:16.677827 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=90654) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 10:32:16.678065 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=90654) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 10:32:16.678160 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=90653) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 10:32:16.678290 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=90654) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 10:32:16.678374 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=90653) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 10:32:16.678515 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=90654) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 10:32:16.678597 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=90653) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 10:32:16.678727 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=90653) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 10:32:16.678817 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=90654) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 10:32:16.678945 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=90653) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 10:32:16.679028 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=90654) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 10:32:16.679154 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=90653) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 10:32:16.679245 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=90654) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 10:32:16.679362 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=90653) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 10:32:16.690284 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=90654) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 10:32:16.690734 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=90653) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 10:32:16.703621 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=90654) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 10:32:16.703728 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=90653) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 10:32:16.703807 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=90654) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 10:32:16.703893 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=90653) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 10:32:16.704024 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=90654) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 10:32:16.704107 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=90653) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 10:32:16.704249 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=90654) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 10:32:16.704331 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=90653) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 10:32:16.704469 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=90654) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 10:32:16.704553 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=90653) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 10:32:16.704686 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=90654) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 10:32:16.704768 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=90653) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 10:32:16.704894 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=90653) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 10:32:16.704984 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=90654) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 10:32:16.722067 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.722275 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 11 10:32:16.722363 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.722499 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 11 10:32:16.722616 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.722815 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 11 10:32:16.722896 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.723031 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 11 10:32:16.723165 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.723311 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.723404 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.723613 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.723694 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.723869 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.723947 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.724128 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.734494 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.734592 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.734774 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.734855 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.735048 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.735142 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.735331 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.735412 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.742293 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=90653) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 10:32:16.742378 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=90654) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 10:32:16.745767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=90654) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 10:32:16.746053 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.746136 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=90653) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 10:32:16.746340 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.746422 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.746623 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.746716 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.746907 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.746989 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.747169 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.747306 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.747476 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.747559 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.747711 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.747793 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.747983 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.748073 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.748253 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.748334 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.748522 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.748601 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.748787 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.771236 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.771653 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.772724 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=90654) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 10:32:16.773011 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.773117 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=90653) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 10:32:16.773404 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.778374 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=90654) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 10:32:16.778662 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.778897 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=90653) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 10:32:16.778977 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.779183 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.779501 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.785289 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=90654) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 10:32:16.785578 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.785783 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=90653) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 10:32:16.786066 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.791808 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=90654) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 10:32:16.792107 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.792387 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.792467 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=90653) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 10:32:16.792674 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.792770 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.792954 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.793035 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.793230 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.793312 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.793503 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.793585 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.793772 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.793852 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.794039 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.794305 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.809463 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.809741 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.810194 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.810469 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.813614 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=90654) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 10:32:16.814284 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=90653) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 10:32:16.816874 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.817175 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.817474 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.817767 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.821402 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=90654) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 10:32:16.821918 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=90653) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 10:32:16.824626 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=90654) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 10:32:16.825337 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=90653) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 10:32:16.827805 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.828108 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.828397 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.828522 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.828683 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.828827 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.828967 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.829100 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.829229 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.829357 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.829598 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.829878 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.845402 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=90654) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 10:32:16.845689 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.845958 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.846224 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.846420 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=90653) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 10:32:16.846709 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.846977 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.847241 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.853260 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.854644 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.854823 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.856228 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.856361 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.856591 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.857740 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.858016 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.859292 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=90654) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 10:32:16.859611 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.859897 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.860187 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.860587 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.860724 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=90653) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 10:32:16.861010 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.861292 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.861573 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.861967 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.872268 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=90654) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 10:32:16.872555 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.872841 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.873126 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.873516 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.873799 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.873927 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=90653) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 10:32:16.874096 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.874223 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.874407 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.874686 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.875064 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.875350 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.875660 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.889664 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=90654) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 10:32:16.891258 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=90653) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 10:32:16.892612 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=90654) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 10:32:16.894173 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=90653) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 10:32:16.895642 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.897076 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.897223 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.898549 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=90654) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 10:32:16.898676 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.898833 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.899231 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.900062 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=90653) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 10:32:16.900967 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.901545 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.906236 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.907713 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=90654) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 10:32:16.907922 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.909382 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=90653) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 10:32:16.910592 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.910990 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.911278 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.911578 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.912276 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.912672 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.912959 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.913240 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.919417 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=90654) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 10:32:16.919731 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.920016 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.921313 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=90653) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 10:32:16.921603 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.921886 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.926911 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=90654) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 10:32:16.927199 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.927505 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.927785 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.928056 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.928340 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.928617 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.928789 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=90653) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 10:32:16.928918 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.929074 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.929201 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.929356 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.929491 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.929628 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.929767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.929898 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.930033 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.930165 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.930317 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.930450 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.930575 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=90654) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.930701 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.930940 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.931220 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.932102 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.932565 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.932951 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.933255 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=90653) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 10:32:16.965848 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55befe8079c0 pid: 90654 (default app) Sep 11 10:32:16.968059 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55befe8079c0 pid: 90653 (default app) Sep 11 10:32:16.969071 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c95e2fd-c221-4fb3-ba2f-4d7d96e73560 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:16.969360 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c95e2fd-c221-4fb3-ba2f-4d7d96e73560 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:16.969634 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c95e2fd-c221-4fb3-ba2f-4d7d96e73560 None None] PATH_INFO: `/v3` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:16.973787 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1/1] 10.222.0.15 () {50 vars in 791 bytes} [Sun Sep 11 10:32:16 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:32:23.757054 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ab58c-03c1-491c-b1d2-9c2ab246b67d None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:23.757544 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ab58c-03c1-491c-b1d2-9c2ab246b67d None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:23.757544 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ab58c-03c1-491c-b1d2-9c2ab246b67d None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:23.762427 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1/2] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:32:23 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 10:32:23.768070 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1649ae66-ebde-4c23-b6f8-0a520a2c9595 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:23.768571 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1649ae66-ebde-4c23-b6f8-0a520a2c9595 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:23.768666 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1649ae66-ebde-4c23-b6f8-0a520a2c9595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:23.840025 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: INFO dbcounter [None req-1649ae66-ebde-4c23-b6f8-0a520a2c9595 None None] Registered counter for database keystone Sep 11 10:32:23.859494 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG oslo_db.sqlalchemy.engines [None req-1649ae66-ebde-4c23-b6f8-0a520a2c9595 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=90654) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 11 10:32:23.865639 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writer thread running {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 11 10:32:24.047463 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG passlib.handlers.bcrypt [None req-1649ae66-ebde-4c23-b6f8-0a520a2c9595 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=90654) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 11 10:32:24.047863 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG passlib.handlers.bcrypt [None req-1649ae66-ebde-4c23-b6f8-0a520a2c9595 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=90654) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 11 10:32:24.105810 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-1649ae66-ebde-4c23-b6f8-0a520a2c9595 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:32:24.107409 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-1649ae66-ebde-4c23-b6f8-0a520a2c9595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:32:24.237393 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 2/3] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:32:23 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 470 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 11 10:32:24.534628 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44ab58c-03c1-491c-b1d2-9c2ab246b67d None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:24.604533 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: INFO dbcounter [None req-f44ab58c-03c1-491c-b1d2-9c2ab246b67d None None] Registered counter for database keystone Sep 11 10:32:24.625888 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG oslo_db.sqlalchemy.engines [None req-f44ab58c-03c1-491c-b1d2-9c2ab246b67d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=90653) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 11 10:32:24.632116 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writer thread running {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 11 10:32:24.750297 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44ab58c-03c1-491c-b1d2-9c2ab246b67d None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:24.754505 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede82e43-bb70-4db3-958c-ef293bdd7f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:24.755129 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede82e43-bb70-4db3-958c-ef293bdd7f68 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:24.755408 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede82e43-bb70-4db3-958c-ef293bdd7f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:24.755704 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede82e43-bb70-4db3-958c-ef293bdd7f68 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:24.763935 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede82e43-bb70-4db3-958c-ef293bdd7f68 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:24.815365 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ede82e43-bb70-4db3-958c-ef293bdd7f68 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 11 10:32:24.815729 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede82e43-bb70-4db3-958c-ef293bdd7f68 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:24.825494 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 2/4] 10.222.0.15 () {58 vars in 1324 bytes} [Sun Sep 11 10:32:24 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 11 10:32:24.831382 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1649ae66-ebde-4c23-b6f8-0a520a2c9595 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:24.848503 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1649ae66-ebde-4c23-b6f8-0a520a2c9595 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:24.853177 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fdc779-8b97-4e34-9e4d-fe1a3abd63ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:24.854276 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fdc779-8b97-4e34-9e4d-fe1a3abd63ea admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:24.854609 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fdc779-8b97-4e34-9e4d-fe1a3abd63ea admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:24.854835 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fdc779-8b97-4e34-9e4d-fe1a3abd63ea admin admin] PATH_INFO: `/v3/projects/dcf7c8cb1c844da09226c671da5a163e` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:24.856459 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fdc779-8b97-4e34-9e4d-fe1a3abd63ea admin admin] RBAC: Authorizing `identity:get_project(project_id=dcf7c8cb1c844da09226c671da5a163e)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:24.909686 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: WARNING keystone.common.rbac_enforcer.enforcer [None req-c5fdc779-8b97-4e34-9e4d-fe1a3abd63ea admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 11 10:32:24.910033 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fdc779-8b97-4e34-9e4d-fe1a3abd63ea admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:24.914532 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 3/5] 10.222.0.15 () {56 vars in 1207 bytes} [Sun Sep 11 10:32:24 2022] GET /identity/v3/projects/dcf7c8cb1c844da09226c671da5a163e => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:32:27.032846 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc9d7c-ecfe-465f-b5a6-78c76bb3cbcd None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:27.033406 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc9d7c-ecfe-465f-b5a6-78c76bb3cbcd None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:27.033406 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc9d7c-ecfe-465f-b5a6-78c76bb3cbcd None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:27.037493 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 3/6] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:32:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 10:32:27.043372 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d3bf8e-7ffe-474e-8bae-962437fe6f26 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:27.043838 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d3bf8e-7ffe-474e-8bae-962437fe6f26 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:27.043954 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d3bf8e-7ffe-474e-8bae-962437fe6f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:27.064235 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-97d3bf8e-7ffe-474e-8bae-962437fe6f26 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:32:27.065758 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-97d3bf8e-7ffe-474e-8bae-962437fe6f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:32:27.073574 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 4/7] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:32:27 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 11 10:32:27.343134 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fc9d7c-ecfe-465f-b5a6-78c76bb3cbcd None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:27.355965 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fc9d7c-ecfe-465f-b5a6-78c76bb3cbcd None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:27.360115 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0848d083-48ae-4a95-bf3b-f240addc962a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:27.360716 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0848d083-48ae-4a95-bf3b-f240addc962a admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:27.360998 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0848d083-48ae-4a95-bf3b-f240addc962a admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:27.361270 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0848d083-48ae-4a95-bf3b-f240addc962a admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:27.368836 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0848d083-48ae-4a95-bf3b-f240addc962a admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:27.370593 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0848d083-48ae-4a95-bf3b-f240addc962a admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:27.379234 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 4/8] 10.222.0.15 () {58 vars in 1324 bytes} [Sun Sep 11 10:32:27 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 10:32:27.384535 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d3bf8e-7ffe-474e-8bae-962437fe6f26 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:27.393929 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d3bf8e-7ffe-474e-8bae-962437fe6f26 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:27.398276 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4c4882-cb31-48d8-8bd4-c9edd9526ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:27.398902 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4c4882-cb31-48d8-8bd4-c9edd9526ab0 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:27.399184 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4c4882-cb31-48d8-8bd4-c9edd9526ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:27.399492 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4c4882-cb31-48d8-8bd4-c9edd9526ab0 admin admin] PATH_INFO: `/v3/users/3570ee53f3c442848938999c912a848b` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:27.401555 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4c4882-cb31-48d8-8bd4-c9edd9526ab0 admin admin] RBAC: Authorizing `identity:get_user(user_id=3570ee53f3c442848938999c912a848b)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:27.403382 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4c4882-cb31-48d8-8bd4-c9edd9526ab0 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:27.404971 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 5/9] 10.222.0.15 () {56 vars in 1198 bytes} [Sun Sep 11 10:32:27 2022] GET /identity/v3/users/3570ee53f3c442848938999c912a848b => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:32:29.613142 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b44a0ff-3806-4601-956e-43abf78caf17 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:29.613642 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b44a0ff-3806-4601-956e-43abf78caf17 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:29.613642 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b44a0ff-3806-4601-956e-43abf78caf17 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:29.614921 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 5/10] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:32:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:32:29.620891 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64c2dc1-257e-4aa0-b26b-590a47a251f1 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:29.621343 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64c2dc1-257e-4aa0-b26b-590a47a251f1 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:29.621439 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64c2dc1-257e-4aa0-b26b-590a47a251f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:29.639763 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-d64c2dc1-257e-4aa0-b26b-590a47a251f1 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:32:29.640961 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-d64c2dc1-257e-4aa0-b26b-590a47a251f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:32:29.654204 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb63ef9-7462-4148-93e4-1ad20a6c6cff None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:29.654492 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb63ef9-7462-4148-93e4-1ad20a6c6cff None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:29.654841 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb63ef9-7462-4148-93e4-1ad20a6c6cff None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:29.656197 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 6/11] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:32:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 10:32:29.661113 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d5d573-7223-42d3-a3bc-cc1d7e7ba353 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:29.661396 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d5d573-7223-42d3-a3bc-cc1d7e7ba353 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:29.661668 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d5d573-7223-42d3-a3bc-cc1d7e7ba353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:29.703298 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG passlib.handlers.bcrypt [None req-66d5d573-7223-42d3-a3bc-cc1d7e7ba353 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=90653) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 11 10:32:29.703717 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG passlib.handlers.bcrypt [None req-66d5d573-7223-42d3-a3bc-cc1d7e7ba353 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=90653) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 11 10:32:29.723265 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 6/12] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:32:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 10:32:29.753182 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-66d5d573-7223-42d3-a3bc-cc1d7e7ba353 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:32:29.754747 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-66d5d573-7223-42d3-a3bc-cc1d7e7ba353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:32:29.853454 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 7/13] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:32:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 10:32:29.995864 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64c2dc1-257e-4aa0-b26b-590a47a251f1 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:30.021213 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:30.100537 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64c2dc1-257e-4aa0-b26b-590a47a251f1 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:30.120554 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d5d573-7223-42d3-a3bc-cc1d7e7ba353 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:30.133598 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6498ecc5-cbba-4d88-ada9-d21d4baea585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:30.134727 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6498ecc5-cbba-4d88-ada9-d21d4baea585 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:30.135014 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6498ecc5-cbba-4d88-ada9-d21d4baea585 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:30.135304 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6498ecc5-cbba-4d88-ada9-d21d4baea585 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:30.146653 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6498ecc5-cbba-4d88-ada9-d21d4baea585 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:32:30.156322 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-6498ecc5-cbba-4d88-ada9-d21d4baea585 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 10:32:30.156322 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:32:30.156322 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:32:30.156322 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:32:30.156322 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:32:30.156322 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:32:30.156322 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:32:30.156322 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:32:30.156322 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:32:30.156322 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:32:30.156322 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:32:30.156322 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:32:30.156322 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:32:30.156322 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:32:30.156322 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:32:30.156322 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:32:30.157251 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:32:30.157251 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:32:30.157251 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:32:30.157251 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:32:30.157251 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:32:30.157251 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:32:30.157251 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:32:30.157251 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:32:30.157251 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:32:30.157251 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:32:30.157251 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:32:30.157251 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:32:30.157251 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:32:30.157251 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:32:30.157251 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:32:30.157251 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:32:30.158285 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:32:30.158285 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:32:30.158285 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:32:30.158285 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:32:30.158285 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:32:30.158285 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:32:30.158285 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:32:30.158285 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:32:30.158285 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:32:30.158285 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:32:30.158285 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 10:32:30.158285 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:32:30.158285 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 7/14] 10.222.0.15 () {56 vars in 1096 bytes} [Sun Sep 11 10:32:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:32:30.163182 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6498ecc5-cbba-4d88-ada9-d21d4baea585 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:30.223345 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d5d573-7223-42d3-a3bc-cc1d7e7ba353 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:30.226667 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:30.243438 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:30.263058 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6498ecc5-cbba-4d88-ada9-d21d4baea585 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:30.263214 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ada9dd3-9852-45a3-bf98-f5f2c94b9516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:30.263917 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ada9dd3-9852-45a3-bf98-f5f2c94b9516 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:30.264204 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ada9dd3-9852-45a3-bf98-f5f2c94b9516 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:30.264481 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ada9dd3-9852-45a3-bf98-f5f2c94b9516 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:30.297277 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc88af87-5ba9-43ba-bcb8-c72130420659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:30.297893 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc88af87-5ba9-43ba-bcb8-c72130420659 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:30.298180 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc88af87-5ba9-43ba-bcb8-c72130420659 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:30.298452 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc88af87-5ba9-43ba-bcb8-c72130420659 None admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:30.299115 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc88af87-5ba9-43ba-bcb8-c72130420659 None admin] RBAC: Adding query filter params (name=admin) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:32:30.299415 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc88af87-5ba9-43ba-bcb8-c72130420659 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:30.300833 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc88af87-5ba9-43ba-bcb8-c72130420659 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:30.314445 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 8/15] 10.222.0.15 () {56 vars in 1099 bytes} [Sun Sep 11 10:32:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:32:30.320078 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc88af87-5ba9-43ba-bcb8-c72130420659 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:30.333969 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ada9dd3-9852-45a3-bf98-f5f2c94b9516 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:30.335518 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ada9dd3-9852-45a3-bf98-f5f2c94b9516 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:30.420897 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc88af87-5ba9-43ba-bcb8-c72130420659 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:30.435110 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 8/16] 10.222.0.15 () {58 vars in 1282 bytes} [Sun Sep 11 10:32:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 11 10:32:30.440625 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ada9dd3-9852-45a3-bf98-f5f2c94b9516 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:30.443319 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:30.453700 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72451277-6af4-40bc-92ae-dcbe7582fe28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:30.454340 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72451277-6af4-40bc-92ae-dcbe7582fe28 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:30.454622 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72451277-6af4-40bc-92ae-dcbe7582fe28 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:30.454895 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72451277-6af4-40bc-92ae-dcbe7582fe28 None admin] PATH_INFO: `/v3/users/3570ee53f3c442848938999c912a848b` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:30.456058 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72451277-6af4-40bc-92ae-dcbe7582fe28 None admin] RBAC: Authorizing `identity:get_user(user_id=3570ee53f3c442848938999c912a848b)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:30.457506 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72451277-6af4-40bc-92ae-dcbe7582fe28 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:30.459231 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 9/17] 10.222.0.15 () {56 vars in 1177 bytes} [Sun Sep 11 10:32:30 2022] GET /identity/v3/users/3570ee53f3c442848938999c912a848b => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:32:30.465073 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72451277-6af4-40bc-92ae-dcbe7582fe28 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:30.480570 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:30.538692 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ada9dd3-9852-45a3-bf98-f5f2c94b9516 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:30.565003 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72451277-6af4-40bc-92ae-dcbe7582fe28 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:30.573865 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d98f38d-cc2d-4fa8-bb99-8fec015a32a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:30.574719 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d98f38d-cc2d-4fa8-bb99-8fec015a32a6 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:30.575001 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d98f38d-cc2d-4fa8-bb99-8fec015a32a6 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:30.575273 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d98f38d-cc2d-4fa8-bb99-8fec015a32a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:30.593375 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d98f38d-cc2d-4fa8-bb99-8fec015a32a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:30.594844 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d98f38d-cc2d-4fa8-bb99-8fec015a32a6 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:30.598217 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638f09f1-a23a-466b-8c7d-f49fe4b730a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:30.598846 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638f09f1-a23a-466b-8c7d-f49fe4b730a9 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:30.599127 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638f09f1-a23a-466b-8c7d-f49fe4b730a9 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:30.599400 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638f09f1-a23a-466b-8c7d-f49fe4b730a9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:30.600531 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638f09f1-a23a-466b-8c7d-f49fe4b730a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:30.602235 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638f09f1-a23a-466b-8c7d-f49fe4b730a9 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:30.603857 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 10/18] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:32:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:32:30.609526 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638f09f1-a23a-466b-8c7d-f49fe4b730a9 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:30.613338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-7d98f38d-cc2d-4fa8-bb99-8fec015a32a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:32:30.613338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:32:30.613338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:32:30.613338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:32:30.613338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:32:30.613338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:32:30.613338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:32:30.613338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:32:30.613338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:32:30.613338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:32:30.613338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:32:30.613338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:32:30.613338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:32:30.613338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:32:30.613338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:32:30.613338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:32:30.614738 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:32:30.614738 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:32:30.614738 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:32:30.614738 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:32:30.614738 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:32:30.614738 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:32:30.614738 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:32:30.614738 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:32:30.614738 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:32:30.614738 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:32:30.614738 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:32:30.614738 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:32:30.614738 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:32:30.614738 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:32:30.614738 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:32:30.614738 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:32:30.614738 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:32:30.615761 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:32:30.615761 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:32:30.615761 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:32:30.615761 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 9/19] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:32:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 10:32:30.622544 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d98f38d-cc2d-4fa8-bb99-8fec015a32a6 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:30.678154 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:30.690836 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:30.711381 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638f09f1-a23a-466b-8c7d-f49fe4b730a9 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:30.724001 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d98f38d-cc2d-4fa8-bb99-8fec015a32a6 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:30.744850 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81b01d3-6191-4b6f-9600-ac4c8c0ba775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:30.745459 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b01d3-6191-4b6f-9600-ac4c8c0ba775 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:30.745745 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b01d3-6191-4b6f-9600-ac4c8c0ba775 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:30.746028 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b01d3-6191-4b6f-9600-ac4c8c0ba775 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:30.746746 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81b01d3-6191-4b6f-9600-ac4c8c0ba775 None admin] RBAC: Adding query filter params (role.id=2cb7d73f0ea8414296367573f6902340, scope.domain.id=default, user.id=3570ee53f3c442848938999c912a848b) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:32:30.747053 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81b01d3-6191-4b6f-9600-ac4c8c0ba775 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:30.748542 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81b01d3-6191-4b6f-9600-ac4c8c0ba775 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:30.755031 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 11/20] 10.222.0.15 () {56 vars in 1322 bytes} [Sun Sep 11 10:32:30 2022] GET /identity/v3/role_assignments?user.id=3570ee53f3c442848938999c912a848b&scope.domain.id=default&role.id=2cb7d73f0ea8414296367573f6902340 => generated 233 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:32:30.756772 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f372e20-ba09-45a7-9835-2fb4493eb86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:30.757382 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f372e20-ba09-45a7-9835-2fb4493eb86f None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:30.757662 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f372e20-ba09-45a7-9835-2fb4493eb86f None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:30.757931 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f372e20-ba09-45a7-9835-2fb4493eb86f None admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:30.758573 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f372e20-ba09-45a7-9835-2fb4493eb86f None admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:32:30.758873 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f372e20-ba09-45a7-9835-2fb4493eb86f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:30.761254 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f372e20-ba09-45a7-9835-2fb4493eb86f None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:30.783503 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 10/21] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:32:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:32:33.393132 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f162409b-82d4-4c72-b3d6-7523e8046757 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:33.393641 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f162409b-82d4-4c72-b3d6-7523e8046757 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:33.393641 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f162409b-82d4-4c72-b3d6-7523e8046757 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:33.395001 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 12/22] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:32:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:32:33.409221 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c09495-793b-41e7-b487-5d2aebacec7b None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:33.409783 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c09495-793b-41e7-b487-5d2aebacec7b None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:33.409783 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c09495-793b-41e7-b487-5d2aebacec7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:33.437813 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-53c09495-793b-41e7-b487-5d2aebacec7b None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:32:33.439030 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-53c09495-793b-41e7-b487-5d2aebacec7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:32:33.533961 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 11/23] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:32:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:32:33.546566 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2500167b-a58c-4364-8bd7-600fbb44d05a None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:33.547045 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2500167b-a58c-4364-8bd7-600fbb44d05a None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:33.547482 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2500167b-a58c-4364-8bd7-600fbb44d05a None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:33.548978 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 13/24] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:32:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:32:33.554361 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc4ab4-aa00-4f45-afe3-a73c9c7b2ad8 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:33.554646 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc4ab4-aa00-4f45-afe3-a73c9c7b2ad8 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:33.554924 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc4ab4-aa00-4f45-afe3-a73c9c7b2ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:33.584528 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-b4fc4ab4-aa00-4f45-afe3-a73c9c7b2ad8 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:32:33.585641 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-b4fc4ab4-aa00-4f45-afe3-a73c9c7b2ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:32:33.639911 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:33.668439 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 12/25] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:32:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:32:33.670512 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3698587-94f9-402a-8641-0a1bcec25d5f None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:33.670800 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3698587-94f9-402a-8641-0a1bcec25d5f None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:33.671070 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3698587-94f9-402a-8641-0a1bcec25d5f None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:33.672566 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 14/26] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:32:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 10:32:33.677679 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dabdd3b-61f6-419d-8935-badb6e3a0ca0 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:33.678030 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dabdd3b-61f6-419d-8935-badb6e3a0ca0 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:33.678305 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dabdd3b-61f6-419d-8935-badb6e3a0ca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:33.697563 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-9dabdd3b-61f6-419d-8935-badb6e3a0ca0 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:32:33.698793 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-9dabdd3b-61f6-419d-8935-badb6e3a0ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:32:33.777442 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 13/27] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:32:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 10:32:33.968663 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3698587-94f9-402a-8641-0a1bcec25d5f None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:34.077811 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dabdd3b-61f6-419d-8935-badb6e3a0ca0 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:34.096722 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3698587-94f9-402a-8641-0a1bcec25d5f None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:34.160770 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b75b80-ad5a-4677-9d9c-766f73073256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:34.161677 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b75b80-ad5a-4677-9d9c-766f73073256 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:34.162246 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b75b80-ad5a-4677-9d9c-766f73073256 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:34.162792 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b75b80-ad5a-4677-9d9c-766f73073256 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:34.187518 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b75b80-ad5a-4677-9d9c-766f73073256 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:32:34.193607 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-c1b75b80-ad5a-4677-9d9c-766f73073256 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 10:32:34.193607 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:32:34.193607 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:32:34.193607 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:32:34.193607 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:32:34.193607 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:32:34.193607 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:32:34.193607 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:32:34.193607 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:32:34.193607 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:32:34.193607 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:32:34.193607 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:32:34.193607 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:32:34.193607 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:32:34.193607 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:32:34.193607 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:32:34.194633 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:32:34.194633 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:32:34.194633 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:32:34.194633 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:32:34.194633 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:32:34.194633 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:32:34.194633 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:32:34.194633 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:32:34.194633 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:32:34.194633 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:32:34.194633 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:32:34.194633 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:32:34.194633 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:32:34.194633 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:32:34.194633 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:32:34.194633 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:32:34.195627 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:32:34.195627 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:32:34.195627 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:32:34.195627 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:32:34.195627 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:32:34.195627 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:32:34.195627 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:32:34.195627 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:32:34.195627 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:32:34.195627 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:32:34.195627 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 10:32:34.195627 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:32:34.204066 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 15/28] 10.222.0.15 () {56 vars in 1096 bytes} [Sun Sep 11 10:32:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:32:34.209994 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b75b80-ad5a-4677-9d9c-766f73073256 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:34.219849 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:34.293730 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dabdd3b-61f6-419d-8935-badb6e3a0ca0 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:34.324021 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:34.362248 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b75b80-ad5a-4677-9d9c-766f73073256 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:34.364268 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5ae679-b1b9-4d21-bf52-4e3dee10916e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:34.364888 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ae679-b1b9-4d21-bf52-4e3dee10916e None admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:34.365168 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ae679-b1b9-4d21-bf52-4e3dee10916e None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:34.365439 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ae679-b1b9-4d21-bf52-4e3dee10916e None admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:34.366109 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5ae679-b1b9-4d21-bf52-4e3dee10916e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:34.367525 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5ae679-b1b9-4d21-bf52-4e3dee10916e None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:34.414436 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa01323a-e537-40cc-9ed8-357d0cc72e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:34.416099 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa01323a-e537-40cc-9ed8-357d0cc72e94 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:34.416615 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa01323a-e537-40cc-9ed8-357d0cc72e94 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:34.417086 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa01323a-e537-40cc-9ed8-357d0cc72e94 None admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:34.418003 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa01323a-e537-40cc-9ed8-357d0cc72e94 None admin] RBAC: Adding query filter params (name=admin) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:32:34.418541 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa01323a-e537-40cc-9ed8-357d0cc72e94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:34.420155 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa01323a-e537-40cc-9ed8-357d0cc72e94 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:34.429466 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 16/29] 10.222.0.15 () {56 vars in 1099 bytes} [Sun Sep 11 10:32:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:32:34.430825 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa01323a-e537-40cc-9ed8-357d0cc72e94 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:34.501847 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:34.520281 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 14/30] 10.222.0.15 () {60 vars in 1131 bytes} [Sun Sep 11 10:32:34 2022] POST /identity/v3/roles => generated 219 bytes in 443 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 10:32:34.521765 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2330c7f-1693-49bc-9b7d-cca731691f49 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:34.522327 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2330c7f-1693-49bc-9b7d-cca731691f49 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:34.522890 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2330c7f-1693-49bc-9b7d-cca731691f49 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:34.524412 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 15/31] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:32:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:32:34.525632 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2330c7f-1693-49bc-9b7d-cca731691f49 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:34.549481 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa01323a-e537-40cc-9ed8-357d0cc72e94 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:34.595078 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2445fd9-acea-49e6-9e96-c487dbee7f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:34.596221 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2445fd9-acea-49e6-9e96-c487dbee7f8d None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:34.596970 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2445fd9-acea-49e6-9e96-c487dbee7f8d None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:34.597421 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2445fd9-acea-49e6-9e96-c487dbee7f8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:34.612627 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2445fd9-acea-49e6-9e96-c487dbee7f8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:34.614482 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2445fd9-acea-49e6-9e96-c487dbee7f8d None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:34.636819 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-d2445fd9-acea-49e6-9e96-c487dbee7f8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:32:34.636819 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:32:34.636819 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:32:34.636819 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:32:34.636819 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:32:34.636819 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:32:34.636819 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:32:34.636819 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:32:34.636819 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:32:34.636819 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:32:34.636819 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:32:34.636819 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:32:34.636819 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:32:34.636819 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:32:34.636819 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:32:34.636819 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:32:34.637746 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:32:34.637746 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:32:34.637746 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:32:34.637746 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:32:34.637746 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:32:34.637746 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:32:34.637746 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:32:34.637746 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:32:34.637746 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:32:34.637746 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:32:34.637746 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:32:34.637746 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:32:34.637746 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:32:34.637746 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:32:34.637746 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:32:34.637746 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:32:34.637746 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:32:34.638584 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:32:34.638584 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:32:34.638584 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:32:34.640484 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 17/32] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:32:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:32:34.642215 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cd5bdd-4988-48b8-bc88-bafbc697492f None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:34.643238 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cd5bdd-4988-48b8-bc88-bafbc697492f None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:34.643745 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cd5bdd-4988-48b8-bc88-bafbc697492f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:34.667040 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-03cd5bdd-4988-48b8-bc88-bafbc697492f None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:32:34.669351 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-03cd5bdd-4988-48b8-bc88-bafbc697492f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:32:34.691402 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2330c7f-1693-49bc-9b7d-cca731691f49 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:34.775460 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23f8cd1-4539-46b5-be83-4a9f6b23b556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:34.775460 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23f8cd1-4539-46b5-be83-4a9f6b23b556 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:34.775460 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23f8cd1-4539-46b5-be83-4a9f6b23b556 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:34.775460 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23f8cd1-4539-46b5-be83-4a9f6b23b556 None admin] PATH_INFO: `/v3/users/3570ee53f3c442848938999c912a848b` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:34.776081 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23f8cd1-4539-46b5-be83-4a9f6b23b556 None admin] RBAC: Authorizing `identity:get_user(user_id=3570ee53f3c442848938999c912a848b)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:34.777806 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23f8cd1-4539-46b5-be83-4a9f6b23b556 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:34.779767 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 16/33] 10.222.0.15 () {56 vars in 1177 bytes} [Sun Sep 11 10:32:34 2022] GET /identity/v3/users/3570ee53f3c442848938999c912a848b => generated 247 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:32:34.780568 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23f8cd1-4539-46b5-be83-4a9f6b23b556 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:34.805480 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 18/34] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:32:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:32:34.807394 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cd5bdd-4988-48b8-bc88-bafbc697492f None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:34.843071 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:34.896142 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:34.929822 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23f8cd1-4539-46b5-be83-4a9f6b23b556 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:34.961657 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cd5bdd-4988-48b8-bc88-bafbc697492f None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:34.969860 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa4f030-70cd-424b-8711-41bb8e89f4c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:34.970597 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa4f030-70cd-424b-8711-41bb8e89f4c3 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:34.970884 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa4f030-70cd-424b-8711-41bb8e89f4c3 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:34.971160 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa4f030-70cd-424b-8711-41bb8e89f4c3 None admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:34.971844 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa4f030-70cd-424b-8711-41bb8e89f4c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:32:34.972325 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa4f030-70cd-424b-8711-41bb8e89f4c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:34.973826 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa4f030-70cd-424b-8711-41bb8e89f4c3 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:34.987452 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 17/35] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:32:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:32:34.988984 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762c5b0-8a6e-4c02-8415-8e2bb6446735 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:34.989283 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762c5b0-8a6e-4c02-8415-8e2bb6446735 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:34.989566 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762c5b0-8a6e-4c02-8415-8e2bb6446735 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:34.991159 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 18/36] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:32:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:32:34.992563 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46943a3-47ed-4f38-b067-6335a4573554 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:34.992842 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46943a3-47ed-4f38-b067-6335a4573554 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:34.993123 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46943a3-47ed-4f38-b067-6335a4573554 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:35.000699 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 19/37] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:32:34 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:32:35.000699 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46943a3-47ed-4f38-b067-6335a4573554 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:35.005452 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48d6daf-bc92-4d4c-b8b8-0e3295d880b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:35.006309 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d6daf-bc92-4d4c-b8b8-0e3295d880b2 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:35.006769 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d6daf-bc92-4d4c-b8b8-0e3295d880b2 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:35.007238 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d6daf-bc92-4d4c-b8b8-0e3295d880b2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:35.008585 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48d6daf-bc92-4d4c-b8b8-0e3295d880b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:35.010272 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48d6daf-bc92-4d4c-b8b8-0e3295d880b2 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:35.012070 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 19/38] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:32:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:32:35.013432 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf19b2-8254-443c-bffe-cfe1f998cb54 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:35.013981 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf19b2-8254-443c-bffe-cfe1f998cb54 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:35.014517 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf19b2-8254-443c-bffe-cfe1f998cb54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:35.035717 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-8baf19b2-8254-443c-bffe-cfe1f998cb54 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:32:35.037260 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-8baf19b2-8254-443c-bffe-cfe1f998cb54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:32:35.110408 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 20/39] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:32:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:32:35.117690 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8535001f-cfce-470b-bcf8-46e00c7e7799 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:35.117690 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8535001f-cfce-470b-bcf8-46e00c7e7799 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:35.117690 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8535001f-cfce-470b-bcf8-46e00c7e7799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:35.117901 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:35.126731 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46943a3-47ed-4f38-b067-6335a4573554 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:35.133071 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-8535001f-cfce-470b-bcf8-46e00c7e7799 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:32:35.135065 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-8535001f-cfce-470b-bcf8-46e00c7e7799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:32:35.175654 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc165b16-dcec-45fe-b4b1-dcb692b51579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:35.176277 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc165b16-dcec-45fe-b4b1-dcb692b51579 None admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:35.176558 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc165b16-dcec-45fe-b4b1-dcb692b51579 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:35.177084 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc165b16-dcec-45fe-b4b1-dcb692b51579 None admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:35.178592 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc165b16-dcec-45fe-b4b1-dcb692b51579 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:35.180079 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc165b16-dcec-45fe-b4b1-dcb692b51579 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:35.185359 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:35.223236 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 21/40] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:32:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:32:35.223236 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8535001f-cfce-470b-bcf8-46e00c7e7799 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:35.230976 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 20/41] 10.222.0.15 () {60 vars in 1141 bytes} [Sun Sep 11 10:32:34 2022] POST /identity/v3/projects => generated 301 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:32:35.232434 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc165b16-dcec-45fe-b4b1-dcb692b51579 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:35.344342 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8535001f-cfce-470b-bcf8-46e00c7e7799 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:35.365714 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc165b16-dcec-45fe-b4b1-dcb692b51579 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:35.388066 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d7b694-472d-4278-94ba-55fe0f1fee51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:35.389713 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d7b694-472d-4278-94ba-55fe0f1fee51 None admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:35.390455 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d7b694-472d-4278-94ba-55fe0f1fee51 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:35.390925 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d7b694-472d-4278-94ba-55fe0f1fee51 None admin] PATH_INFO: `/v3/domains/default/users/3570ee53f3c442848938999c912a848b/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:35.393980 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d7b694-472d-4278-94ba-55fe0f1fee51 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=3570ee53f3c442848938999c912a848b, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:35.396413 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d7b694-472d-4278-94ba-55fe0f1fee51 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:35.411953 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41abfca-c813-4bfa-a2ad-d93e63e252ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:35.414461 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41abfca-c813-4bfa-a2ad-d93e63e252ad None admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:35.420475 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41abfca-c813-4bfa-a2ad-d93e63e252ad None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:35.421541 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41abfca-c813-4bfa-a2ad-d93e63e252ad None admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:35.423638 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41abfca-c813-4bfa-a2ad-d93e63e252ad None admin] RBAC: Authorizing `identity:create_role()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:35.427810 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41abfca-c813-4bfa-a2ad-d93e63e252ad None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:35.458750 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 21/42] 10.222.0.15 () {60 vars in 1131 bytes} [Sun Sep 11 10:32:35 2022] POST /identity/v3/roles => generated 225 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:32:35.485093 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41abfca-c813-4bfa-a2ad-d93e63e252ad None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:35.503968 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:35.634288 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41abfca-c813-4bfa-a2ad-d93e63e252ad None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:35.675463 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 22/43] 10.222.0.15 () {58 vars in 1361 bytes} [Sun Sep 11 10:32:35 2022] PUT /identity/v3/domains/default/users/3570ee53f3c442848938999c912a848b/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:32:35.675463 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d7b694-472d-4278-94ba-55fe0f1fee51 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:35.676067 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4950c5-8bd2-421a-b226-a59be76868b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:35.676067 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4950c5-8bd2-421a-b226-a59be76868b2 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:35.676067 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4950c5-8bd2-421a-b226-a59be76868b2 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:35.676067 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4950c5-8bd2-421a-b226-a59be76868b2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:35.677589 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4950c5-8bd2-421a-b226-a59be76868b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:35.679177 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4950c5-8bd2-421a-b226-a59be76868b2 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:35.680897 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 22/44] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:32:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:32:35.687444 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4950c5-8bd2-421a-b226-a59be76868b2 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:35.718937 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:35.803039 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:35.812629 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d7b694-472d-4278-94ba-55fe0f1fee51 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:35.834337 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4950c5-8bd2-421a-b226-a59be76868b2 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:35.852699 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856f64da-2ff3-4124-8bdc-1274b2b55575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:35.854383 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856f64da-2ff3-4124-8bdc-1274b2b55575 None admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:35.854383 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856f64da-2ff3-4124-8bdc-1274b2b55575 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:35.854383 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856f64da-2ff3-4124-8bdc-1274b2b55575 None admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:35.855001 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856f64da-2ff3-4124-8bdc-1274b2b55575 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:35.856642 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856f64da-2ff3-4124-8bdc-1274b2b55575 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:35.878628 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 23/45] 10.222.0.15 () {60 vars in 1131 bytes} [Sun Sep 11 10:32:35 2022] POST /identity/v3/roles => generated 223 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:32:35.878802 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7e4492-cbca-4f1e-ab73-a985aa36c79e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:35.880059 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e4492-cbca-4f1e-ab73-a985aa36c79e None admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:35.880560 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e4492-cbca-4f1e-ab73-a985aa36c79e None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:35.881031 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e4492-cbca-4f1e-ab73-a985aa36c79e None admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:35.881908 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7e4492-cbca-4f1e-ab73-a985aa36c79e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:35.883634 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7e4492-cbca-4f1e-ab73-a985aa36c79e None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:35.914954 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 23/46] 10.222.0.15 () {60 vars in 1141 bytes} [Sun Sep 11 10:32:35 2022] POST /identity/v3/projects => generated 312 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 10:32:37.926758 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b9631a-05b0-4da1-9142-84a9d96ffd9d None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:37.927309 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b9631a-05b0-4da1-9142-84a9d96ffd9d None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:37.927309 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b9631a-05b0-4da1-9142-84a9d96ffd9d None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:37.928653 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 24/47] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:32:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:32:37.935097 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d69ea5-59e0-4861-a9b3-588b550d4e96 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:37.935538 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d69ea5-59e0-4861-a9b3-588b550d4e96 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:37.935691 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d69ea5-59e0-4861-a9b3-588b550d4e96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:37.954298 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-48d69ea5-59e0-4861-a9b3-588b550d4e96 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:32:37.955835 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-48d69ea5-59e0-4861-a9b3-588b550d4e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:32:38.026968 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 24/48] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:32:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:32:38.095095 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e29e9c5-1d0a-41ad-b584-cc944fb82bc0 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:38.095388 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e29e9c5-1d0a-41ad-b584-cc944fb82bc0 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:38.095701 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e29e9c5-1d0a-41ad-b584-cc944fb82bc0 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:38.097049 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 25/49] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:32:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:32:38.102476 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ac51c-c5fd-4916-8dba-666bf3e7c898 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:38.102763 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ac51c-c5fd-4916-8dba-666bf3e7c898 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:38.103186 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ac51c-c5fd-4916-8dba-666bf3e7c898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:38.121659 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-a48ac51c-c5fd-4916-8dba-666bf3e7c898 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:32:38.122866 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-a48ac51c-c5fd-4916-8dba-666bf3e7c898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:32:38.170898 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:38.188140 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 25/50] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:32:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:32:38.296728 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e29e9c5-1d0a-41ad-b584-cc944fb82bc0 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:38.397457 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:38.405329 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e29e9c5-1d0a-41ad-b584-cc944fb82bc0 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:38.439106 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881c642c-4155-45d9-8602-3552c1af39e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:38.439679 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881c642c-4155-45d9-8602-3552c1af39e5 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:38.439964 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881c642c-4155-45d9-8602-3552c1af39e5 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:38.440241 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881c642c-4155-45d9-8602-3552c1af39e5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:38.450197 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881c642c-4155-45d9-8602-3552c1af39e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:32:38.455041 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-881c642c-4155-45d9-8602-3552c1af39e5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 10:32:38.455041 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:32:38.455041 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:32:38.455041 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:32:38.455041 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:32:38.455041 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:32:38.455041 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:32:38.455041 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:32:38.455041 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:32:38.455041 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:32:38.455041 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:32:38.455041 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:32:38.455041 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:32:38.455041 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:32:38.455041 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:32:38.455041 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:32:38.456067 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:32:38.456067 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:32:38.456067 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:32:38.456067 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:32:38.456067 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:32:38.456067 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:32:38.456067 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:32:38.456067 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:32:38.456067 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:32:38.456067 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:32:38.456067 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:32:38.456067 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:32:38.456067 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:32:38.456067 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:32:38.456067 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:32:38.456067 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:32:38.456861 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:32:38.456861 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:32:38.456861 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:32:38.456861 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:32:38.456861 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:32:38.456861 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:32:38.456861 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:32:38.456861 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:32:38.456861 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:32:38.456861 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:32:38.456861 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 10:32:38.456861 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:32:38.456861 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 26/51] 10.222.0.15 () {56 vars in 1096 bytes} [Sun Sep 11 10:32:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:32:38.462046 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48ac51c-c5fd-4916-8dba-666bf3e7c898 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:38.465951 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881c642c-4155-45d9-8602-3552c1af39e5 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:38.563682 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48ac51c-c5fd-4916-8dba-666bf3e7c898 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:38.564855 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881c642c-4155-45d9-8602-3552c1af39e5 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:38.597681 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250347e0-85cf-46ae-931c-03b63ddc9423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:38.598307 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250347e0-85cf-46ae-931c-03b63ddc9423 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:38.598588 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250347e0-85cf-46ae-931c-03b63ddc9423 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:38.598859 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250347e0-85cf-46ae-931c-03b63ddc9423 None admin] PATH_INFO: `/v3/domains/default` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:38.599042 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b08a402-ffd3-44d3-b619-0812afeac58f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:38.599681 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08a402-ffd3-44d3-b619-0812afeac58f None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:38.599966 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08a402-ffd3-44d3-b619-0812afeac58f None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:38.600050 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250347e0-85cf-46ae-931c-03b63ddc9423 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:38.600243 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08a402-ffd3-44d3-b619-0812afeac58f None admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:38.600906 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b08a402-ffd3-44d3-b619-0812afeac58f None admin] RBAC: Adding query filter params (name=admin) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:32:38.601469 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b08a402-ffd3-44d3-b619-0812afeac58f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:38.601747 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250347e0-85cf-46ae-931c-03b63ddc9423 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:38.602898 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b08a402-ffd3-44d3-b619-0812afeac58f None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:38.603514 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 27/52] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:32:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:32:38.609116 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250347e0-85cf-46ae-931c-03b63ddc9423 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:38.620079 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 26/53] 10.222.0.15 () {56 vars in 1099 bytes} [Sun Sep 11 10:32:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:32:38.621082 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b08a402-ffd3-44d3-b619-0812afeac58f None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:38.635287 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:38.718177 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250347e0-85cf-46ae-931c-03b63ddc9423 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:38.752545 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52355912-ab2c-4dde-a792-518b1765fc0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:38.753154 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52355912-ab2c-4dde-a792-518b1765fc0d None admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:38.753440 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52355912-ab2c-4dde-a792-518b1765fc0d None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:38.753710 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52355912-ab2c-4dde-a792-518b1765fc0d None admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:38.754357 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52355912-ab2c-4dde-a792-518b1765fc0d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:38.755841 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52355912-ab2c-4dde-a792-518b1765fc0d None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:38.774331 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:38.783873 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 28/54] 10.222.0.15 () {60 vars in 1140 bytes} [Sun Sep 11 10:32:38 2022] POST /identity/v3/projects => generated 298 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 10:32:38.830476 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b08a402-ffd3-44d3-b619-0812afeac58f None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:38.867217 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08164631-eabf-473f-9de0-6c1f9bfbd228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:38.867898 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08164631-eabf-473f-9de0-6c1f9bfbd228 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:38.868179 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08164631-eabf-473f-9de0-6c1f9bfbd228 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:38.868527 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08164631-eabf-473f-9de0-6c1f9bfbd228 None admin] PATH_INFO: `/v3/users/3570ee53f3c442848938999c912a848b` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:38.869883 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08164631-eabf-473f-9de0-6c1f9bfbd228 None admin] RBAC: Authorizing `identity:get_user(user_id=3570ee53f3c442848938999c912a848b)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:38.871372 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08164631-eabf-473f-9de0-6c1f9bfbd228 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:38.873057 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 27/55] 10.222.0.15 () {56 vars in 1177 bytes} [Sun Sep 11 10:32:38 2022] GET /identity/v3/users/3570ee53f3c442848938999c912a848b => generated 247 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:32:38.878745 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52355912-ab2c-4dde-a792-518b1765fc0d None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:38.959134 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:38.983744 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52355912-ab2c-4dde-a792-518b1765fc0d None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:39.018918 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43543db7-68c3-4f46-ae5f-0e6525a80a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:39.019581 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43543db7-68c3-4f46-ae5f-0e6525a80a8b None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:39.019863 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43543db7-68c3-4f46-ae5f-0e6525a80a8b None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:39.020133 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43543db7-68c3-4f46-ae5f-0e6525a80a8b None admin] PATH_INFO: `/v3/domains/default` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:39.021224 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43543db7-68c3-4f46-ae5f-0e6525a80a8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:39.022666 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43543db7-68c3-4f46-ae5f-0e6525a80a8b None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:39.024259 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 29/56] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:32:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:32:39.030804 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08164631-eabf-473f-9de0-6c1f9bfbd228 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:39.129654 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08164631-eabf-473f-9de0-6c1f9bfbd228 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:39.146631 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:39.164810 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcae1f98-459b-471a-a14e-43a9f56ebfdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:39.165974 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae1f98-459b-471a-a14e-43a9f56ebfdb None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:39.166216 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae1f98-459b-471a-a14e-43a9f56ebfdb None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:39.166583 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae1f98-459b-471a-a14e-43a9f56ebfdb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:39.167311 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcae1f98-459b-471a-a14e-43a9f56ebfdb None admin] RBAC: Adding query filter params (role.id=2cb7d73f0ea8414296367573f6902340, scope.domain.id=default, user.id=3570ee53f3c442848938999c912a848b) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:32:39.167651 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcae1f98-459b-471a-a14e-43a9f56ebfdb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:39.169103 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcae1f98-459b-471a-a14e-43a9f56ebfdb None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:39.178102 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 28/57] 10.222.0.15 () {56 vars in 1322 bytes} [Sun Sep 11 10:32:39 2022] GET /identity/v3/role_assignments?user.id=3570ee53f3c442848938999c912a848b&scope.domain.id=default&role.id=2cb7d73f0ea8414296367573f6902340 => generated 530 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:32:40.930817 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9454f7-d66c-4df9-ac59-41b4425d3b1c None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:40.932045 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9454f7-d66c-4df9-ac59-41b4425d3b1c None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:40.932045 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9454f7-d66c-4df9-ac59-41b4425d3b1c None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:40.933370 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 30/58] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:32:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:32:40.938449 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620eb0ae-5500-41ba-bb79-41a4c0ce13af None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:40.938813 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620eb0ae-5500-41ba-bb79-41a4c0ce13af None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:40.939011 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620eb0ae-5500-41ba-bb79-41a4c0ce13af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:40.956784 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-620eb0ae-5500-41ba-bb79-41a4c0ce13af None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:32:40.957973 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-620eb0ae-5500-41ba-bb79-41a4c0ce13af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:32:41.022353 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 29/59] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:32:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:32:41.288987 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9454f7-d66c-4df9-ac59-41b4425d3b1c None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:41.387645 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9454f7-d66c-4df9-ac59-41b4425d3b1c None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:41.420064 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9cd005-bd20-4cf3-b11a-e502299eeff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:41.420695 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9cd005-bd20-4cf3-b11a-e502299eeff4 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:41.420977 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9cd005-bd20-4cf3-b11a-e502299eeff4 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:41.421254 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9cd005-bd20-4cf3-b11a-e502299eeff4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:41.422347 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9cd005-bd20-4cf3-b11a-e502299eeff4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:41.423823 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9cd005-bd20-4cf3-b11a-e502299eeff4 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:41.425373 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 31/60] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:32:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:32:41.430803 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620eb0ae-5500-41ba-bb79-41a4c0ce13af None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:41.527675 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620eb0ae-5500-41ba-bb79-41a4c0ce13af None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:41.537886 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:41.561560 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c899c9ac-300d-4aef-8841-48c58558ba7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:41.562103 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c899c9ac-300d-4aef-8841-48c58558ba7c None admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:41.562389 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c899c9ac-300d-4aef-8841-48c58558ba7c None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:41.562664 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c899c9ac-300d-4aef-8841-48c58558ba7c None admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:41.563320 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c899c9ac-300d-4aef-8841-48c58558ba7c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:41.564807 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c899c9ac-300d-4aef-8841-48c58558ba7c None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:41.591132 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 30/61] 10.222.0.15 () {60 vars in 1132 bytes} [Sun Sep 11 10:32:41 2022] POST /identity/v3/users => generated 275 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:32:44.953377 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdccb902-afc1-4440-bb31-51033b2eaa30 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:44.954405 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdccb902-afc1-4440-bb31-51033b2eaa30 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:44.954881 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdccb902-afc1-4440-bb31-51033b2eaa30 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:44.956738 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 32/62] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:32:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:32:44.961939 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba4ef80-f7cb-446f-88d8-b1374dcffb0c None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:44.962976 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba4ef80-f7cb-446f-88d8-b1374dcffb0c None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:44.963496 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba4ef80-f7cb-446f-88d8-b1374dcffb0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:44.968195 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e7dd73-213a-4310-9aad-fe389a80fcfb None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:44.968764 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e7dd73-213a-4310-9aad-fe389a80fcfb None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:44.969250 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e7dd73-213a-4310-9aad-fe389a80fcfb None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:44.970788 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 33/63] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:32:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:32:44.976411 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b413c55-84f3-4d17-ad36-52eb591d9112 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:44.976958 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b413c55-84f3-4d17-ad36-52eb591d9112 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:44.977459 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b413c55-84f3-4d17-ad36-52eb591d9112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:44.992831 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-eba4ef80-f7cb-446f-88d8-b1374dcffb0c None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:32:44.994228 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-eba4ef80-f7cb-446f-88d8-b1374dcffb0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:32:44.997329 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-6b413c55-84f3-4d17-ad36-52eb591d9112 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:32:44.999049 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-6b413c55-84f3-4d17-ad36-52eb591d9112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:32:45.007300 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:45.112543 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 31/64] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:32:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:32:45.133724 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 34/65] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:32:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 10:32:45.265840 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e2683-5107-43ab-9126-2eec6a535d7a None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:45.266302 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e2683-5107-43ab-9126-2eec6a535d7a None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:45.266810 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e2683-5107-43ab-9126-2eec6a535d7a None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:45.268395 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 32/66] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:32:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:32:45.273808 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad1041-8a24-400d-a94d-34b81bf54836 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:45.274100 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad1041-8a24-400d-a94d-34b81bf54836 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:45.274383 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad1041-8a24-400d-a94d-34b81bf54836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:45.303697 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-69ad1041-8a24-400d-a94d-34b81bf54836 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:32:45.304893 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-69ad1041-8a24-400d-a94d-34b81bf54836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:32:45.376248 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067fa6cc-5eae-475e-a9ad-bc2583d05e60 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:45.376915 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067fa6cc-5eae-475e-a9ad-bc2583d05e60 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:45.377420 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067fa6cc-5eae-475e-a9ad-bc2583d05e60 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:45.379040 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 33/67] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:32:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 10:32:45.386611 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7f5b39-28da-43ed-8330-b77bcf1f4886 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:45.387628 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7f5b39-28da-43ed-8330-b77bcf1f4886 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:45.388242 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7f5b39-28da-43ed-8330-b77bcf1f4886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:45.406799 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 35/68] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:32:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:32:45.410124 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-3e7f5b39-28da-43ed-8330-b77bcf1f4886 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:32:45.411735 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-3e7f5b39-28da-43ed-8330-b77bcf1f4886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:32:45.484432 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 34/69] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:32:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:32:45.506216 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ad1041-8a24-400d-a94d-34b81bf54836 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:45.555714 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c3070-0401-43a5-9725-09665b244c94 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:45.556823 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c3070-0401-43a5-9725-09665b244c94 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:45.557319 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c3070-0401-43a5-9725-09665b244c94 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:45.558981 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 35/70] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:32:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 10:32:45.564221 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d9e5c4-a9bc-4b92-938f-25710d662222 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:45.564732 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d9e5c4-a9bc-4b92-938f-25710d662222 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:45.565323 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d9e5c4-a9bc-4b92-938f-25710d662222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:45.586895 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-a8d9e5c4-a9bc-4b92-938f-25710d662222 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:32:45.589229 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-a8d9e5c4-a9bc-4b92-938f-25710d662222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:32:45.599455 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:45.620409 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:45.632787 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ad1041-8a24-400d-a94d-34b81bf54836 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:45.671251 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc938411-a0c0-4ab2-8178-2d75c5b08ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:45.671995 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc938411-a0c0-4ab2-8178-2d75c5b08ff2 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:45.672283 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc938411-a0c0-4ab2-8178-2d75c5b08ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:45.672638 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc938411-a0c0-4ab2-8178-2d75c5b08ff2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:45.678461 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 36/71] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:32:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 10:32:45.679301 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d9e5c4-a9bc-4b92-938f-25710d662222 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:45.688592 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc938411-a0c0-4ab2-8178-2d75c5b08ff2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:32:45.699267 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-fc938411-a0c0-4ab2-8178-2d75c5b08ff2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 10:32:45.699267 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:32:45.699267 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:32:45.699267 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:32:45.699267 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:32:45.699267 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:32:45.699267 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:32:45.699267 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:32:45.699267 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:32:45.699267 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:32:45.699267 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:32:45.699267 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:32:45.699267 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:32:45.699267 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:32:45.699267 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:32:45.699267 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:32:45.700173 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:32:45.700173 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:32:45.700173 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:32:45.700173 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:32:45.700173 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:32:45.700173 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:32:45.700173 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:32:45.700173 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:32:45.700173 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:32:45.700173 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:32:45.700173 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:32:45.700173 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:32:45.700173 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:32:45.700173 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:32:45.700173 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:32:45.700173 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:32:45.700929 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:32:45.700929 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:32:45.700929 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:32:45.700929 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:32:45.700929 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:32:45.700929 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:32:45.700929 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:32:45.700929 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:32:45.700929 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:32:45.700929 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:32:45.700929 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 10:32:45.700929 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:32:45.704611 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 36/72] 10.222.0.15 () {56 vars in 1096 bytes} [Sun Sep 11 10:32:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:32:45.710462 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc938411-a0c0-4ab2-8178-2d75c5b08ff2 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:45.793144 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d9e5c4-a9bc-4b92-938f-25710d662222 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:45.829277 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc938411-a0c0-4ab2-8178-2d75c5b08ff2 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:45.829726 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0737a81a-e08c-4527-8e5b-6eb85ef95148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:45.830367 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0737a81a-e08c-4527-8e5b-6eb85ef95148 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:45.830652 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0737a81a-e08c-4527-8e5b-6eb85ef95148 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:45.830923 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0737a81a-e08c-4527-8e5b-6eb85ef95148 None admin] PATH_INFO: `/v3/roles/member` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:45.842416 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0737a81a-e08c-4527-8e5b-6eb85ef95148 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:32:45.848928 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-0737a81a-e08c-4527-8e5b-6eb85ef95148 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 10:32:45.848928 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:32:45.848928 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:32:45.848928 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:32:45.848928 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:32:45.848928 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:32:45.848928 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:32:45.848928 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:32:45.848928 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:32:45.848928 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:32:45.848928 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:32:45.848928 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:32:45.848928 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:32:45.848928 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:32:45.848928 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:32:45.848928 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:32:45.849749 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:32:45.849749 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:32:45.849749 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:32:45.849749 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:32:45.849749 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:32:45.849749 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:32:45.849749 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:32:45.849749 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:32:45.849749 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:32:45.849749 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:32:45.849749 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:32:45.849749 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:32:45.849749 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:32:45.849749 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:32:45.849749 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:32:45.849749 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:32:45.856795 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:32:45.856795 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:32:45.856795 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:32:45.856795 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:32:45.856795 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:32:45.856795 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:32:45.856795 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:32:45.856795 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:32:45.856795 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:32:45.856795 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:32:45.856795 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 10:32:45.856795 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:32:45.856795 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 37/73] 10.222.0.15 () {56 vars in 1099 bytes} [Sun Sep 11 10:32:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:32:45.856795 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0737a81a-e08c-4527-8e5b-6eb85ef95148 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:45.865957 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:45.872987 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b2c8b7-4bd9-4979-83d1-1d2763bae149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:45.873635 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b2c8b7-4bd9-4979-83d1-1d2763bae149 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:45.873886 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b2c8b7-4bd9-4979-83d1-1d2763bae149 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:45.874161 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b2c8b7-4bd9-4979-83d1-1d2763bae149 None admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:45.874446 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:45.874826 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b2c8b7-4bd9-4979-83d1-1d2763bae149 None admin] RBAC: Adding query filter params (name=admin) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:32:45.875128 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b2c8b7-4bd9-4979-83d1-1d2763bae149 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:45.876563 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b2c8b7-4bd9-4979-83d1-1d2763bae149 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:45.885334 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 37/74] 10.222.0.15 () {56 vars in 1099 bytes} [Sun Sep 11 10:32:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:32:45.885969 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b2c8b7-4bd9-4979-83d1-1d2763bae149 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:45.969336 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0737a81a-e08c-4527-8e5b-6eb85ef95148 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:45.989643 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b2c8b7-4bd9-4979-83d1-1d2763bae149 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:46.004229 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c0589d-cea4-4850-86bc-d135b72e2d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:46.004857 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c0589d-cea4-4850-86bc-d135b72e2d4b None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:46.005133 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c0589d-cea4-4850-86bc-d135b72e2d4b None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:46.005403 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c0589d-cea4-4850-86bc-d135b72e2d4b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:46.016439 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c0589d-cea4-4850-86bc-d135b72e2d4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:32:46.021229 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-90c0589d-cea4-4850-86bc-d135b72e2d4b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 10:32:46.021229 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:32:46.021229 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:32:46.021229 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:32:46.021229 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:32:46.021229 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:32:46.021229 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:32:46.021229 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:32:46.021229 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:32:46.021229 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:32:46.021229 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:32:46.021229 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:32:46.021229 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:32:46.021229 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:32:46.021229 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:32:46.021229 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:32:46.022065 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:32:46.022065 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:32:46.022065 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:32:46.022065 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:32:46.022065 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:32:46.022065 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:32:46.022065 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:32:46.022065 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:32:46.022065 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:32:46.022065 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:32:46.022065 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:32:46.022065 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:32:46.022065 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:32:46.022065 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:32:46.022065 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:32:46.022065 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:32:46.023071 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:32:46.023071 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:32:46.023071 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:32:46.023071 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:32:46.023071 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:32:46.023071 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:32:46.023071 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:32:46.023071 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:32:46.023071 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:32:46.023071 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:32:46.023071 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 10:32:46.023071 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:32:46.023071 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 38/75] 10.222.0.15 () {56 vars in 1114 bytes} [Sun Sep 11 10:32:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:32:46.023071 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c0589d-cea4-4850-86bc-d135b72e2d4b None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:46.028129 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6315ed1-550b-486a-8b44-06918b372e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:46.028760 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6315ed1-550b-486a-8b44-06918b372e3f None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:46.029036 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6315ed1-550b-486a-8b44-06918b372e3f None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:46.029305 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6315ed1-550b-486a-8b44-06918b372e3f None admin] PATH_INFO: `/v3/roles/member` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:46.039636 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6315ed1-550b-486a-8b44-06918b372e3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:32:46.044944 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-b6315ed1-550b-486a-8b44-06918b372e3f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 10:32:46.044944 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:32:46.044944 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:32:46.044944 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:32:46.044944 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:32:46.044944 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:32:46.044944 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:32:46.044944 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:32:46.044944 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:32:46.044944 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:32:46.044944 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:32:46.044944 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:32:46.044944 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:32:46.044944 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:32:46.044944 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:32:46.044944 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:32:46.045827 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:32:46.045827 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:32:46.045827 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:32:46.045827 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:32:46.045827 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:32:46.045827 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:32:46.045827 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:32:46.045827 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:32:46.045827 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:32:46.045827 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:32:46.045827 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:32:46.045827 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:32:46.045827 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:32:46.045827 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:32:46.045827 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:32:46.045827 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:32:46.046603 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:32:46.046603 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:32:46.046603 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:32:46.046603 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:32:46.046603 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:32:46.046603 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:32:46.046603 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:32:46.046603 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:32:46.046603 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:32:46.046603 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:32:46.046603 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 10:32:46.046603 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:32:46.046603 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 38/76] 10.222.0.15 () {56 vars in 1099 bytes} [Sun Sep 11 10:32:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:32:46.047298 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6315ed1-550b-486a-8b44-06918b372e3f None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:46.101483 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:46.106017 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:46.128818 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c0589d-cea4-4850-86bc-d135b72e2d4b None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:46.145298 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6315ed1-550b-486a-8b44-06918b372e3f None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:46.162479 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee34022b-a6e7-4293-a9af-0fd16d23adcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:46.163087 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee34022b-a6e7-4293-a9af-0fd16d23adcb None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:46.163367 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee34022b-a6e7-4293-a9af-0fd16d23adcb None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:46.163668 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee34022b-a6e7-4293-a9af-0fd16d23adcb None admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:46.164596 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee34022b-a6e7-4293-a9af-0fd16d23adcb None admin] RBAC: Adding query filter params (name=member) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:32:46.164900 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee34022b-a6e7-4293-a9af-0fd16d23adcb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:46.166948 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee34022b-a6e7-4293-a9af-0fd16d23adcb None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:46.174710 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 39/77] 10.222.0.15 () {56 vars in 1101 bytes} [Sun Sep 11 10:32:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:32:46.175608 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee34022b-a6e7-4293-a9af-0fd16d23adcb None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:46.183185 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065c3714-3711-404a-8550-4caa133bc789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:46.183866 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065c3714-3711-404a-8550-4caa133bc789 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:46.184148 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065c3714-3711-404a-8550-4caa133bc789 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:46.184424 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065c3714-3711-404a-8550-4caa133bc789 None admin] PATH_INFO: `/v3/users/3570ee53f3c442848938999c912a848b` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:46.185544 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065c3714-3711-404a-8550-4caa133bc789 None admin] RBAC: Authorizing `identity:get_user(user_id=3570ee53f3c442848938999c912a848b)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:46.186997 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065c3714-3711-404a-8550-4caa133bc789 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:46.193063 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 39/78] 10.222.0.15 () {56 vars in 1177 bytes} [Sun Sep 11 10:32:46 2022] GET /identity/v3/users/3570ee53f3c442848938999c912a848b => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:32:46.193063 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065c3714-3711-404a-8550-4caa133bc789 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:46.281064 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee34022b-a6e7-4293-a9af-0fd16d23adcb None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:46.288760 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065c3714-3711-404a-8550-4caa133bc789 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:46.314336 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:46.318082 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d79564-ba13-45ac-8804-a14015019360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:46.318723 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d79564-ba13-45ac-8804-a14015019360 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:46.319003 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d79564-ba13-45ac-8804-a14015019360 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:46.319285 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d79564-ba13-45ac-8804-a14015019360 None admin] PATH_INFO: `/v3/domains/default` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:46.320456 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d79564-ba13-45ac-8804-a14015019360 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:46.322005 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d79564-ba13-45ac-8804-a14015019360 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:46.323597 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 40/79] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:32:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:32:46.324356 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d79564-ba13-45ac-8804-a14015019360 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:46.331229 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bf780d-9fe4-4f93-8df7-3d5e30c3750d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:46.331887 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bf780d-9fe4-4f93-8df7-3d5e30c3750d None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:46.332178 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bf780d-9fe4-4f93-8df7-3d5e30c3750d None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:46.332452 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bf780d-9fe4-4f93-8df7-3d5e30c3750d None admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:46.333056 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:46.333255 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bf780d-9fe4-4f93-8df7-3d5e30c3750d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:32:46.333560 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bf780d-9fe4-4f93-8df7-3d5e30c3750d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:46.335079 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bf780d-9fe4-4f93-8df7-3d5e30c3750d None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:46.344093 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 40/80] 10.222.0.15 () {56 vars in 1111 bytes} [Sun Sep 11 10:32:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:32:46.344877 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bf780d-9fe4-4f93-8df7-3d5e30c3750d None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:46.430249 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d79564-ba13-45ac-8804-a14015019360 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:46.447572 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bf780d-9fe4-4f93-8df7-3d5e30c3750d None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:46.464469 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ae2090-a362-490e-b1ec-83a7cfe7f274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:46.465074 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae2090-a362-490e-b1ec-83a7cfe7f274 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:46.465355 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae2090-a362-490e-b1ec-83a7cfe7f274 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:46.465632 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae2090-a362-490e-b1ec-83a7cfe7f274 None admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:46.466289 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ae2090-a362-490e-b1ec-83a7cfe7f274 None admin] RBAC: Adding query filter params (name=member) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:32:46.466594 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ae2090-a362-490e-b1ec-83a7cfe7f274 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:46.467994 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ae2090-a362-490e-b1ec-83a7cfe7f274 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:46.475956 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 41/81] 10.222.0.15 () {56 vars in 1101 bytes} [Sun Sep 11 10:32:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:32:46.477074 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ae2090-a362-490e-b1ec-83a7cfe7f274 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:46.481355 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1c0892-f8a5-4dc9-a128-a5658f54686d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:46.482068 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c0892-f8a5-4dc9-a128-a5658f54686d None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:46.482351 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c0892-f8a5-4dc9-a128-a5658f54686d None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:46.482620 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c0892-f8a5-4dc9-a128-a5658f54686d None admin] PATH_INFO: `/v3/users/0175a3d1c7824b08b476a3f905aad38e` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:46.496488 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1c0892-f8a5-4dc9-a128-a5658f54686d None admin] RBAC: Authorizing `identity:get_user(user_id=0175a3d1c7824b08b476a3f905aad38e)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:46.497964 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1c0892-f8a5-4dc9-a128-a5658f54686d None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:46.504016 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 41/82] 10.222.0.15 () {56 vars in 1177 bytes} [Sun Sep 11 10:32:46 2022] GET /identity/v3/users/0175a3d1c7824b08b476a3f905aad38e => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:32:46.504016 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1c0892-f8a5-4dc9-a128-a5658f54686d None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:46.543863 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:46.548868 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:46.583213 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ae2090-a362-490e-b1ec-83a7cfe7f274 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:46.599650 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1c0892-f8a5-4dc9-a128-a5658f54686d None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:46.619162 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b67675-5c8a-44f7-8146-56a3d1388fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:46.619814 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b67675-5c8a-44f7-8146-56a3d1388fdf None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:46.620103 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b67675-5c8a-44f7-8146-56a3d1388fdf None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:46.620385 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b67675-5c8a-44f7-8146-56a3d1388fdf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:46.632393 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70a63ae-ab71-4cb2-974d-22eafa967b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:46.633002 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70a63ae-ab71-4cb2-974d-22eafa967b33 None admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:46.633288 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70a63ae-ab71-4cb2-974d-22eafa967b33 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:46.633568 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70a63ae-ab71-4cb2-974d-22eafa967b33 None admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:46.634797 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70a63ae-ab71-4cb2-974d-22eafa967b33 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:46.636272 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70a63ae-ab71-4cb2-974d-22eafa967b33 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:46.663089 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 42/83] 10.222.0.15 () {60 vars in 1132 bytes} [Sun Sep 11 10:32:46 2022] POST /identity/v3/users => generated 289 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:32:46.663871 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70a63ae-ab71-4cb2-974d-22eafa967b33 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:46.694400 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b67675-5c8a-44f7-8146-56a3d1388fdf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:46.695971 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b67675-5c8a-44f7-8146-56a3d1388fdf None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:46.772081 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:46.787281 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:46.791613 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70a63ae-ab71-4cb2-974d-22eafa967b33 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:46.815754 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 42/84] 10.222.0.15 () {58 vars in 1282 bytes} [Sun Sep 11 10:32:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 10:32:46.822486 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b67675-5c8a-44f7-8146-56a3d1388fdf None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:46.831710 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b71841-99e4-4812-b04d-2baeb97975c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:46.832361 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b71841-99e4-4812-b04d-2baeb97975c1 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:46.832644 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b71841-99e4-4812-b04d-2baeb97975c1 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:46.832921 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b71841-99e4-4812-b04d-2baeb97975c1 None admin] PATH_INFO: `/v3/users/0175a3d1c7824b08b476a3f905aad38e` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:46.834080 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b71841-99e4-4812-b04d-2baeb97975c1 None admin] RBAC: Authorizing `identity:get_user(user_id=0175a3d1c7824b08b476a3f905aad38e)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:46.835578 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b71841-99e4-4812-b04d-2baeb97975c1 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:46.837249 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 43/85] 10.222.0.15 () {56 vars in 1177 bytes} [Sun Sep 11 10:32:46 2022] GET /identity/v3/users/0175a3d1c7824b08b476a3f905aad38e => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:32:46.842076 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b71841-99e4-4812-b04d-2baeb97975c1 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:46.947577 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b67675-5c8a-44f7-8146-56a3d1388fdf None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:46.966164 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b71841-99e4-4812-b04d-2baeb97975c1 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:46.994251 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978f7bae-de71-427d-8353-a70f08bc6c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:46.994944 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f7bae-de71-427d-8353-a70f08bc6c29 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:46.995226 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f7bae-de71-427d-8353-a70f08bc6c29 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:46.995539 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f7bae-de71-427d-8353-a70f08bc6c29 None admin] PATH_INFO: `/v3/users/0175a3d1c7824b08b476a3f905aad38e` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:46.996851 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978f7bae-de71-427d-8353-a70f08bc6c29 None admin] RBAC: Authorizing `identity:get_user(user_id=0175a3d1c7824b08b476a3f905aad38e)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:46.998998 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978f7bae-de71-427d-8353-a70f08bc6c29 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:47.000730 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 43/86] 10.222.0.15 () {56 vars in 1177 bytes} [Sun Sep 11 10:32:46 2022] GET /identity/v3/users/0175a3d1c7824b08b476a3f905aad38e => generated 275 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:32:47.002650 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978f7bae-de71-427d-8353-a70f08bc6c29 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:47.012544 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f104ef8b-9cd7-45e2-80fc-48c6b1e7175f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:47.013205 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f104ef8b-9cd7-45e2-80fc-48c6b1e7175f None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:47.013516 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f104ef8b-9cd7-45e2-80fc-48c6b1e7175f None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:47.013800 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f104ef8b-9cd7-45e2-80fc-48c6b1e7175f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:47.050390 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:47.061486 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:47.093748 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f104ef8b-9cd7-45e2-80fc-48c6b1e7175f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:47.095358 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f104ef8b-9cd7-45e2-80fc-48c6b1e7175f None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:47.144978 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978f7bae-de71-427d-8353-a70f08bc6c29 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:47.182894 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1614d1ec-f4e7-484c-9130-55ed645c82db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:47.183722 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1614d1ec-f4e7-484c-9130-55ed645c82db None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:47.184071 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1614d1ec-f4e7-484c-9130-55ed645c82db None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:47.184416 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1614d1ec-f4e7-484c-9130-55ed645c82db None admin] PATH_INFO: `/v3/projects/1b12f8d03f4f4f4c8b5f37b804de18c5` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:47.185635 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1614d1ec-f4e7-484c-9130-55ed645c82db None admin] RBAC: Authorizing `identity:get_project(project_id=1b12f8d03f4f4f4c8b5f37b804de18c5)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:47.187156 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1614d1ec-f4e7-484c-9130-55ed645c82db None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:47.189198 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 44/87] 10.222.0.15 () {56 vars in 1186 bytes} [Sun Sep 11 10:32:47 2022] GET /identity/v3/projects/1b12f8d03f4f4f4c8b5f37b804de18c5 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:32:47.190113 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1614d1ec-f4e7-484c-9130-55ed645c82db None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:47.225925 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 44/88] 10.222.0.15 () {58 vars in 1282 bytes} [Sun Sep 11 10:32:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 10:32:47.226965 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f104ef8b-9cd7-45e2-80fc-48c6b1e7175f None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:47.300924 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:47.306293 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1614d1ec-f4e7-484c-9130-55ed645c82db None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:47.310016 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:47.341770 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f104ef8b-9cd7-45e2-80fc-48c6b1e7175f None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:47.346218 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8581f46-02e8-45e7-8bdd-2b5107f360c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:47.347002 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8581f46-02e8-45e7-8bdd-2b5107f360c4 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:47.347387 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8581f46-02e8-45e7-8bdd-2b5107f360c4 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:47.347798 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8581f46-02e8-45e7-8bdd-2b5107f360c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:47.378429 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0333414-cc19-41a6-a65f-7e1f0087ff2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:47.379054 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0333414-cc19-41a6-a65f-7e1f0087ff2c None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:47.379340 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0333414-cc19-41a6-a65f-7e1f0087ff2c None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:47.379657 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0333414-cc19-41a6-a65f-7e1f0087ff2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:47.420562 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8581f46-02e8-45e7-8bdd-2b5107f360c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:47.422137 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8581f46-02e8-45e7-8bdd-2b5107f360c4 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:47.454478 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0333414-cc19-41a6-a65f-7e1f0087ff2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:47.456013 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0333414-cc19-41a6-a65f-7e1f0087ff2c None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:47.525877 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 45/89] 10.222.0.15 () {58 vars in 1282 bytes} [Sun Sep 11 10:32:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 10:32:47.526641 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:47.531523 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8581f46-02e8-45e7-8bdd-2b5107f360c4 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:47.535182 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:47.569013 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 45/90] 10.222.0.15 () {58 vars in 1282 bytes} [Sun Sep 11 10:32:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 10:32:47.569889 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0333414-cc19-41a6-a65f-7e1f0087ff2c None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:47.643246 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8581f46-02e8-45e7-8bdd-2b5107f360c4 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:47.684377 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663431b9-e87f-4194-8b15-bf3af7d6ef80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:47.685558 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663431b9-e87f-4194-8b15-bf3af7d6ef80 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:47.686381 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663431b9-e87f-4194-8b15-bf3af7d6ef80 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:47.687144 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663431b9-e87f-4194-8b15-bf3af7d6ef80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:47.688158 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663431b9-e87f-4194-8b15-bf3af7d6ef80 None admin] RBAC: Adding query filter params (role.id=2cb7d73f0ea8414296367573f6902340, scope.project.id=1b12f8d03f4f4f4c8b5f37b804de18c5, user.id=3570ee53f3c442848938999c912a848b) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:32:47.688582 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663431b9-e87f-4194-8b15-bf3af7d6ef80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:47.690191 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663431b9-e87f-4194-8b15-bf3af7d6ef80 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:47.697134 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0333414-cc19-41a6-a65f-7e1f0087ff2c None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:47.697856 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 46/91] 10.222.0.15 () {56 vars in 1374 bytes} [Sun Sep 11 10:32:47 2022] GET /identity/v3/role_assignments?user.id=3570ee53f3c442848938999c912a848b&scope.project.id=1b12f8d03f4f4f4c8b5f37b804de18c5&role.id=2cb7d73f0ea8414296367573f6902340 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:32:47.699969 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663431b9-e87f-4194-8b15-bf3af7d6ef80 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:47.770504 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ceb9f0-306b-4f90-b41f-81dd5335fc8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:47.771165 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ceb9f0-306b-4f90-b41f-81dd5335fc8e None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:47.771463 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ceb9f0-306b-4f90-b41f-81dd5335fc8e None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:47.771745 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ceb9f0-306b-4f90-b41f-81dd5335fc8e None admin] PATH_INFO: `/v3/projects/1b12f8d03f4f4f4c8b5f37b804de18c5` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:47.773496 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ceb9f0-306b-4f90-b41f-81dd5335fc8e None admin] RBAC: Authorizing `identity:get_project(project_id=1b12f8d03f4f4f4c8b5f37b804de18c5)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:47.775681 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ceb9f0-306b-4f90-b41f-81dd5335fc8e None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:47.780582 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 46/92] 10.222.0.15 () {56 vars in 1186 bytes} [Sun Sep 11 10:32:47 2022] GET /identity/v3/projects/1b12f8d03f4f4f4c8b5f37b804de18c5 => generated 298 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:32:47.781420 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ceb9f0-306b-4f90-b41f-81dd5335fc8e None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:47.848417 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:47.849981 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:47.915199 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663431b9-e87f-4194-8b15-bf3af7d6ef80 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:47.937505 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ceb9f0-306b-4f90-b41f-81dd5335fc8e None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:47.976686 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880f0109-3542-4c4a-b625-7b572ec04b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:47.977452 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f0109-3542-4c4a-b625-7b572ec04b9e None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:47.978465 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f0109-3542-4c4a-b625-7b572ec04b9e None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:47.978748 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f0109-3542-4c4a-b625-7b572ec04b9e None admin] PATH_INFO: `/v3/projects/b21540a9dc144b76bf6e2e819b54cccf` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:47.979998 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880f0109-3542-4c4a-b625-7b572ec04b9e None admin] RBAC: Authorizing `identity:get_project(project_id=b21540a9dc144b76bf6e2e819b54cccf)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:47.981473 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880f0109-3542-4c4a-b625-7b572ec04b9e None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:47.983135 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 47/93] 10.222.0.15 () {56 vars in 1186 bytes} [Sun Sep 11 10:32:47 2022] GET /identity/v3/projects/b21540a9dc144b76bf6e2e819b54cccf => generated 312 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:32:47.988347 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880f0109-3542-4c4a-b625-7b572ec04b9e None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:48.000125 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200d2d35-8a91-42d7-961c-4bda5c11640c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:48.000824 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d2d35-8a91-42d7-961c-4bda5c11640c None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:48.001065 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d2d35-8a91-42d7-961c-4bda5c11640c None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:48.001347 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d2d35-8a91-42d7-961c-4bda5c11640c None admin] PATH_INFO: `/v3/projects/1b12f8d03f4f4f4c8b5f37b804de18c5` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:48.002524 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200d2d35-8a91-42d7-961c-4bda5c11640c None admin] RBAC: Authorizing `identity:get_project(project_id=1b12f8d03f4f4f4c8b5f37b804de18c5)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:48.004073 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200d2d35-8a91-42d7-961c-4bda5c11640c None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:48.005733 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 47/94] 10.222.0.15 () {56 vars in 1186 bytes} [Sun Sep 11 10:32:47 2022] GET /identity/v3/projects/1b12f8d03f4f4f4c8b5f37b804de18c5 => generated 298 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:32:48.006495 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200d2d35-8a91-42d7-961c-4bda5c11640c None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:48.157979 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200d2d35-8a91-42d7-961c-4bda5c11640c None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:48.189454 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880f0109-3542-4c4a-b625-7b572ec04b9e None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:48.191185 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:48.218838 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277941f6-90d4-4493-9efc-a5fe1e1e3a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:48.219488 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277941f6-90d4-4493-9efc-a5fe1e1e3a4c None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:48.219780 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277941f6-90d4-4493-9efc-a5fe1e1e3a4c None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:48.220060 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277941f6-90d4-4493-9efc-a5fe1e1e3a4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:48.221220 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277941f6-90d4-4493-9efc-a5fe1e1e3a4c None admin] RBAC: Adding query filter params (role.id=f836487a66c74c05838133c9d0614bbf, scope.project.id=b21540a9dc144b76bf6e2e819b54cccf, user.id=0175a3d1c7824b08b476a3f905aad38e) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:32:48.221536 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277941f6-90d4-4493-9efc-a5fe1e1e3a4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:48.222997 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277941f6-90d4-4493-9efc-a5fe1e1e3a4c None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:48.229756 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 48/95] 10.222.0.15 () {56 vars in 1374 bytes} [Sun Sep 11 10:32:48 2022] GET /identity/v3/role_assignments?user.id=0175a3d1c7824b08b476a3f905aad38e&scope.project.id=b21540a9dc144b76bf6e2e819b54cccf&role.id=f836487a66c74c05838133c9d0614bbf => generated 259 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:32:48.230587 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277941f6-90d4-4493-9efc-a5fe1e1e3a4c None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:48.245147 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:48.348270 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec44f4d-9030-4fb2-b09a-d08e941eca9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:48.348931 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec44f4d-9030-4fb2-b09a-d08e941eca9a None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:48.349220 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec44f4d-9030-4fb2-b09a-d08e941eca9a None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:48.349517 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec44f4d-9030-4fb2-b09a-d08e941eca9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:48.350249 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec44f4d-9030-4fb2-b09a-d08e941eca9a None admin] RBAC: Adding query filter params (role.id=f836487a66c74c05838133c9d0614bbf, scope.project.id=1b12f8d03f4f4f4c8b5f37b804de18c5, user.id=0175a3d1c7824b08b476a3f905aad38e) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:32:48.350560 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec44f4d-9030-4fb2-b09a-d08e941eca9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:48.352355 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec44f4d-9030-4fb2-b09a-d08e941eca9a None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:48.373164 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 48/96] 10.222.0.15 () {56 vars in 1374 bytes} [Sun Sep 11 10:32:47 2022] GET /identity/v3/role_assignments?user.id=0175a3d1c7824b08b476a3f905aad38e&scope.project.id=1b12f8d03f4f4f4c8b5f37b804de18c5&role.id=f836487a66c74c05838133c9d0614bbf => generated 259 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:32:48.382244 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277941f6-90d4-4493-9efc-a5fe1e1e3a4c None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:48.422426 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7826f65a-95ec-4167-b5a3-9fade0744ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:48.422974 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7826f65a-95ec-4167-b5a3-9fade0744ae1 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:48.423254 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7826f65a-95ec-4167-b5a3-9fade0744ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:48.423563 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7826f65a-95ec-4167-b5a3-9fade0744ae1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:48.424306 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7826f65a-95ec-4167-b5a3-9fade0744ae1 None admin] RBAC: Adding query filter params (role.id=cdb3aa1bd85b4694964629ff7cf8039b, scope.project.id=1b12f8d03f4f4f4c8b5f37b804de18c5, user.id=0175a3d1c7824b08b476a3f905aad38e) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:32:48.424616 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7826f65a-95ec-4167-b5a3-9fade0744ae1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:48.426096 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7826f65a-95ec-4167-b5a3-9fade0744ae1 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:48.438173 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 49/97] 10.222.0.15 () {56 vars in 1374 bytes} [Sun Sep 11 10:32:48 2022] GET /identity/v3/role_assignments?user.id=0175a3d1c7824b08b476a3f905aad38e&scope.project.id=1b12f8d03f4f4f4c8b5f37b804de18c5&role.id=cdb3aa1bd85b4694964629ff7cf8039b => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:32:49.457003 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceb578c-1b59-4d4c-9f79-3e612507a092 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:49.457510 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceb578c-1b59-4d4c-9f79-3e612507a092 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:49.457614 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceb578c-1b59-4d4c-9f79-3e612507a092 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:49.458904 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 49/98] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:32:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:32:49.464819 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bece1971-5259-4aac-ba20-ed735fa6b4a6 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:49.465297 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bece1971-5259-4aac-ba20-ed735fa6b4a6 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:49.465370 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bece1971-5259-4aac-ba20-ed735fa6b4a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:49.492545 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-bece1971-5259-4aac-ba20-ed735fa6b4a6 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:32:49.494097 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-bece1971-5259-4aac-ba20-ed735fa6b4a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:32:49.570237 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:49.588021 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 50/99] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:32:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:32:49.596549 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c257bb4-3fac-4412-bdab-35898eacff24 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:49.596841 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c257bb4-3fac-4412-bdab-35898eacff24 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:49.597126 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c257bb4-3fac-4412-bdab-35898eacff24 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:49.598462 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 50/100] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:32:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:32:49.606409 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1411cd-aecd-4c1d-9328-66b2ac72192b None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:49.606693 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1411cd-aecd-4c1d-9328-66b2ac72192b None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:49.606965 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1411cd-aecd-4c1d-9328-66b2ac72192b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:49.632468 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-db1411cd-aecd-4c1d-9328-66b2ac72192b None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:32:49.634585 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-db1411cd-aecd-4c1d-9328-66b2ac72192b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:32:49.709098 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 51/101] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:32:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:32:49.995709 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c257bb4-3fac-4412-bdab-35898eacff24 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:50.126855 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c257bb4-3fac-4412-bdab-35898eacff24 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:50.169788 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16748cd-a3c8-4ca6-b73c-b1b6cdf1f9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:50.170439 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16748cd-a3c8-4ca6-b73c-b1b6cdf1f9f0 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:50.170724 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16748cd-a3c8-4ca6-b73c-b1b6cdf1f9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:50.171003 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16748cd-a3c8-4ca6-b73c-b1b6cdf1f9f0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:50.192436 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16748cd-a3c8-4ca6-b73c-b1b6cdf1f9f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:32:50.197383 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-a16748cd-a3c8-4ca6-b73c-b1b6cdf1f9f0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 10:32:50.197383 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:32:50.197383 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:32:50.197383 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:32:50.197383 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:32:50.197383 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:32:50.197383 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:32:50.197383 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:32:50.197383 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:32:50.197383 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:32:50.197383 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:32:50.197383 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:32:50.197383 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:32:50.197383 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:32:50.197383 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:32:50.197383 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:32:50.198588 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:32:50.198588 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:32:50.198588 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:32:50.198588 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:32:50.198588 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:32:50.198588 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:32:50.198588 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:32:50.198588 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:32:50.198588 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:32:50.198588 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:32:50.198588 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:32:50.198588 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:32:50.198588 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:32:50.198588 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:32:50.198588 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:32:50.198588 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:32:50.199369 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:32:50.199369 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:32:50.199369 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:32:50.199369 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:32:50.199369 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:32:50.199369 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:32:50.199369 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:32:50.199369 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:32:50.199369 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:32:50.199369 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:32:50.199369 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 10:32:50.199369 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:32:50.199369 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 51/102] 10.222.0.15 () {56 vars in 1099 bytes} [Sun Sep 11 10:32:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:32:50.204335 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1411cd-aecd-4c1d-9328-66b2ac72192b None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:50.284213 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16748cd-a3c8-4ca6-b73c-b1b6cdf1f9f0 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:50.363215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1411cd-aecd-4c1d-9328-66b2ac72192b None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:50.385779 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:50.402095 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5437bdf5-5c4e-49d1-9880-7318062a2436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:50.402714 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5437bdf5-5c4e-49d1-9880-7318062a2436 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:50.402996 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5437bdf5-5c4e-49d1-9880-7318062a2436 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:50.403270 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5437bdf5-5c4e-49d1-9880-7318062a2436 None admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:50.403974 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5437bdf5-5c4e-49d1-9880-7318062a2436 None admin] RBAC: Adding query filter params (name=reader) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:32:50.404395 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5437bdf5-5c4e-49d1-9880-7318062a2436 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:50.405800 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5437bdf5-5c4e-49d1-9880-7318062a2436 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:50.408398 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:50.414832 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 52/103] 10.222.0.15 () {56 vars in 1101 bytes} [Sun Sep 11 10:32:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:32:50.421725 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5437bdf5-5c4e-49d1-9880-7318062a2436 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:50.579194 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5437bdf5-5c4e-49d1-9880-7318062a2436 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:50.587983 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16748cd-a3c8-4ca6-b73c-b1b6cdf1f9f0 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:50.661416 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b376a3d-ab23-4ace-bac0-752b0d9efa9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:50.662069 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b376a3d-ab23-4ace-bac0-752b0d9efa9e None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:50.662355 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b376a3d-ab23-4ace-bac0-752b0d9efa9e None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:50.662632 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b376a3d-ab23-4ace-bac0-752b0d9efa9e None admin] PATH_INFO: `/v3/domains/default` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:50.663781 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5fbe87-4e69-4cea-9956-8eeaa2c920a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:50.664420 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5fbe87-4e69-4cea-9956-8eeaa2c920a1 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:50.664700 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5fbe87-4e69-4cea-9956-8eeaa2c920a1 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:50.664972 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5fbe87-4e69-4cea-9956-8eeaa2c920a1 None admin] PATH_INFO: `/v3/users/53e4b2016de34e7bb3adfa8fa78a1610` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:50.666256 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b376a3d-ab23-4ace-bac0-752b0d9efa9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:50.667768 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b376a3d-ab23-4ace-bac0-752b0d9efa9e None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:50.676300 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 52/104] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:32:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:32:50.677130 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b376a3d-ab23-4ace-bac0-752b0d9efa9e None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:50.715342 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5fbe87-4e69-4cea-9956-8eeaa2c920a1 None admin] RBAC: Authorizing `identity:get_user(user_id=53e4b2016de34e7bb3adfa8fa78a1610)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:50.716877 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5fbe87-4e69-4cea-9956-8eeaa2c920a1 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:50.718771 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 53/105] 10.222.0.15 () {56 vars in 1177 bytes} [Sun Sep 11 10:32:50 2022] GET /identity/v3/users/53e4b2016de34e7bb3adfa8fa78a1610 => generated 289 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:32:50.724949 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5fbe87-4e69-4cea-9956-8eeaa2c920a1 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:50.839871 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:50.898668 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:50.911106 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b376a3d-ab23-4ace-bac0-752b0d9efa9e None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:50.962559 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5fbe87-4e69-4cea-9956-8eeaa2c920a1 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:50.988643 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bae52b3-15b7-4618-84cf-801b6b2814c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:50.989628 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bae52b3-15b7-4618-84cf-801b6b2814c8 None admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:50.990129 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bae52b3-15b7-4618-84cf-801b6b2814c8 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:50.990600 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bae52b3-15b7-4618-84cf-801b6b2814c8 None admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:50.991636 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bae52b3-15b7-4618-84cf-801b6b2814c8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:50.993397 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bae52b3-15b7-4618-84cf-801b6b2814c8 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:51.043446 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 53/106] 10.222.0.15 () {60 vars in 1141 bytes} [Sun Sep 11 10:32:50 2022] POST /identity/v3/projects => generated 302 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:32:51.044528 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8820e978-4b8f-493d-b610-c12ce15365ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:51.044652 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8820e978-4b8f-493d-b610-c12ce15365ae None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:51.047445 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8820e978-4b8f-493d-b610-c12ce15365ae None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:51.047445 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8820e978-4b8f-493d-b610-c12ce15365ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:51.097497 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a40101f-8bf4-4093-a38c-d17b6c5ed266 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:51.097783 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a40101f-8bf4-4093-a38c-d17b6c5ed266 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:51.097972 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a40101f-8bf4-4093-a38c-d17b6c5ed266 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:51.099335 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 54/107] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:32:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 10:32:51.106977 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4babe2e-4799-4dca-bd32-034594865b9a None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:51.107274 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4babe2e-4799-4dca-bd32-034594865b9a None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:51.107571 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4babe2e-4799-4dca-bd32-034594865b9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:51.138240 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-f4babe2e-4799-4dca-bd32-034594865b9a None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:32:51.139635 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-f4babe2e-4799-4dca-bd32-034594865b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:32:51.188644 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8820e978-4b8f-493d-b610-c12ce15365ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:51.190133 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8820e978-4b8f-493d-b610-c12ce15365ae None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:51.270141 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 55/108] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:32:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 10:32:51.293168 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:51.369167 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 54/109] 10.222.0.15 () {58 vars in 1282 bytes} [Sun Sep 11 10:32:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 644 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 10:32:51.375003 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4babe2e-4799-4dca-bd32-034594865b9a None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:51.485775 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:51.516799 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4babe2e-4799-4dca-bd32-034594865b9a None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:51.560561 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a49c6f1-47a6-4b5f-adb2-0701250cf26a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:51.561212 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a49c6f1-47a6-4b5f-adb2-0701250cf26a None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:51.561489 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a49c6f1-47a6-4b5f-adb2-0701250cf26a None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:51.561758 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a49c6f1-47a6-4b5f-adb2-0701250cf26a None admin] PATH_INFO: `/v3/projects/1b12f8d03f4f4f4c8b5f37b804de18c5` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:51.568357 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a49c6f1-47a6-4b5f-adb2-0701250cf26a None admin] RBAC: Authorizing `identity:get_project(project_id=1b12f8d03f4f4f4c8b5f37b804de18c5)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:51.569903 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a49c6f1-47a6-4b5f-adb2-0701250cf26a None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:51.573511 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 56/110] 10.222.0.15 () {56 vars in 1186 bytes} [Sun Sep 11 10:32:51 2022] GET /identity/v3/projects/1b12f8d03f4f4f4c8b5f37b804de18c5 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:32:51.579004 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8820e978-4b8f-493d-b610-c12ce15365ae None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:51.631876 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19068fce-34ad-493b-8ff2-e10abc087c43 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:51.632350 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19068fce-34ad-493b-8ff2-e10abc087c43 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:51.632350 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19068fce-34ad-493b-8ff2-e10abc087c43 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:51.634735 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 57/111] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:32:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 10:32:51.655722 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960d8cf5-ee4b-44f3-a8e3-84d9ba082772 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:51.656056 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960d8cf5-ee4b-44f3-a8e3-84d9ba082772 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:51.656390 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960d8cf5-ee4b-44f3-a8e3-84d9ba082772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:51.679627 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-960d8cf5-ee4b-44f3-a8e3-84d9ba082772 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:32:51.681734 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-960d8cf5-ee4b-44f3-a8e3-84d9ba082772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:32:51.733053 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8820e978-4b8f-493d-b610-c12ce15365ae None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:51.773513 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0cc28a-8e85-4748-97ad-a7955cca6271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:51.773708 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 58/112] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:32:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 10:32:51.774537 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960d8cf5-ee4b-44f3-a8e3-84d9ba082772 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:51.775248 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0cc28a-8e85-4748-97ad-a7955cca6271 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:51.775783 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0cc28a-8e85-4748-97ad-a7955cca6271 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:51.776256 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0cc28a-8e85-4748-97ad-a7955cca6271 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:51.777294 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0cc28a-8e85-4748-97ad-a7955cca6271 None admin] RBAC: Adding query filter params (role.id=d577f4ba2f9c412e9530e50d7f3d6bd7, scope.project.id=1b12f8d03f4f4f4c8b5f37b804de18c5, user.id=53e4b2016de34e7bb3adfa8fa78a1610) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:32:51.777905 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0cc28a-8e85-4748-97ad-a7955cca6271 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:51.779584 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0cc28a-8e85-4748-97ad-a7955cca6271 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:51.785951 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 55/113] 10.222.0.15 () {56 vars in 1374 bytes} [Sun Sep 11 10:32:51 2022] GET /identity/v3/role_assignments?user.id=53e4b2016de34e7bb3adfa8fa78a1610&scope.project.id=1b12f8d03f4f4f4c8b5f37b804de18c5&role.id=d577f4ba2f9c412e9530e50d7f3d6bd7 => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:32:51.787666 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0634bd-d46d-4115-92bd-4638de1ecac7 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:51.788168 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0634bd-d46d-4115-92bd-4638de1ecac7 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:51.788658 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0634bd-d46d-4115-92bd-4638de1ecac7 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:51.790656 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 56/114] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:32:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:32:51.810142 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe15660-6313-40bb-b8eb-a87b72b2d0e3 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:51.810810 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe15660-6313-40bb-b8eb-a87b72b2d0e3 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:51.811288 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe15660-6313-40bb-b8eb-a87b72b2d0e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:51.853305 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-2fe15660-6313-40bb-b8eb-a87b72b2d0e3 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:32:51.854543 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-2fe15660-6313-40bb-b8eb-a87b72b2d0e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:32:51.859988 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:51.960252 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:51.984298 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 57/115] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:32:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 10:32:51.985944 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f720137-2e8d-459c-bb5a-be4d94469742 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:51.986349 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f720137-2e8d-459c-bb5a-be4d94469742 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:51.986719 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f720137-2e8d-459c-bb5a-be4d94469742 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:51.988545 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 58/116] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:32:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:32:51.994591 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653be126-520f-4280-8f61-eee2a7af4658 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:51.995073 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653be126-520f-4280-8f61-eee2a7af4658 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:51.995457 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653be126-520f-4280-8f61-eee2a7af4658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:52.017708 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-653be126-520f-4280-8f61-eee2a7af4658 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:32:52.018886 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-653be126-520f-4280-8f61-eee2a7af4658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:32:52.019582 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960d8cf5-ee4b-44f3-a8e3-84d9ba082772 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:52.074337 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c902a47b-4c35-4b69-b1c5-c029f5ab1be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:52.075051 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902a47b-4c35-4b69-b1c5-c029f5ab1be2 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:52.075332 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902a47b-4c35-4b69-b1c5-c029f5ab1be2 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:52.075629 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902a47b-4c35-4b69-b1c5-c029f5ab1be2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:52.093156 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c902a47b-4c35-4b69-b1c5-c029f5ab1be2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:32:52.100823 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-c902a47b-4c35-4b69-b1c5-c029f5ab1be2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 10:32:52.100823 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:32:52.100823 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:32:52.100823 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:32:52.100823 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:32:52.100823 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:32:52.100823 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:32:52.100823 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:32:52.100823 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:32:52.100823 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:32:52.100823 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:32:52.100823 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:32:52.100823 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:32:52.100823 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:32:52.100823 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:32:52.100823 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:32:52.101986 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:32:52.101986 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:32:52.101986 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:32:52.101986 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:32:52.101986 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:32:52.101986 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:32:52.101986 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:32:52.101986 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:32:52.101986 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:32:52.101986 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:32:52.101986 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:32:52.101986 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:32:52.101986 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:32:52.101986 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:32:52.101986 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:32:52.101986 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:32:52.103489 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:32:52.103489 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:32:52.103489 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:32:52.103489 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:32:52.103489 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:32:52.103489 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:32:52.103489 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:32:52.103489 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:32:52.103489 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:32:52.103489 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:32:52.103489 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 10:32:52.103489 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:32:52.104085 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 59/117] 10.222.0.15 () {56 vars in 1096 bytes} [Sun Sep 11 10:32:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:32:52.115511 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c902a47b-4c35-4b69-b1c5-c029f5ab1be2 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:52.133983 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 59/118] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:32:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 10:32:52.202023 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653be126-520f-4280-8f61-eee2a7af4658 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:52.264911 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c902a47b-4c35-4b69-b1c5-c029f5ab1be2 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:52.327309 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddacdb1c-42f4-462e-b918-bb2caab00a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:52.328040 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddacdb1c-42f4-462e-b918-bb2caab00a2a None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:52.328332 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddacdb1c-42f4-462e-b918-bb2caab00a2a None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:52.328617 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddacdb1c-42f4-462e-b918-bb2caab00a2a None admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:52.329295 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddacdb1c-42f4-462e-b918-bb2caab00a2a None admin] RBAC: Adding query filter params (name=admin) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:32:52.329604 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddacdb1c-42f4-462e-b918-bb2caab00a2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:52.331002 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddacdb1c-42f4-462e-b918-bb2caab00a2a None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:52.332640 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:52.345620 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 60/119] 10.222.0.15 () {56 vars in 1099 bytes} [Sun Sep 11 10:32:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:32:52.353643 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddacdb1c-42f4-462e-b918-bb2caab00a2a None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:52.372780 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:52.470715 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653be126-520f-4280-8f61-eee2a7af4658 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:52.522987 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddacdb1c-42f4-462e-b918-bb2caab00a2a None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:52.541602 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fa106e-b83d-44c2-b70f-173fc37293af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:52.542362 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fa106e-b83d-44c2-b70f-173fc37293af None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:52.542640 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fa106e-b83d-44c2-b70f-173fc37293af None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:52.542916 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fa106e-b83d-44c2-b70f-173fc37293af None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:52.554617 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fa106e-b83d-44c2-b70f-173fc37293af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:32:52.561143 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-54fa106e-b83d-44c2-b70f-173fc37293af None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 10:32:52.561143 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:32:52.561143 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:32:52.561143 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:32:52.561143 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:32:52.561143 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:32:52.561143 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:32:52.561143 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:32:52.561143 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:32:52.561143 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:32:52.561143 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:32:52.561143 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:32:52.561143 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:32:52.561143 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:32:52.561143 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:32:52.561143 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:32:52.562192 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:32:52.562192 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:32:52.562192 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:32:52.562192 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:32:52.562192 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:32:52.562192 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:32:52.562192 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:32:52.562192 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:32:52.562192 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:32:52.562192 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:32:52.562192 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:32:52.562192 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:32:52.562192 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:32:52.562192 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:32:52.562192 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:32:52.562192 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:32:52.562988 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:32:52.562988 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:32:52.562988 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:32:52.562988 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:32:52.562988 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:32:52.562988 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:32:52.562988 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:32:52.562988 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:32:52.562988 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:32:52.562988 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:32:52.562988 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 10:32:52.562988 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:32:52.562988 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 60/120] 10.222.0.15 () {56 vars in 1114 bytes} [Sun Sep 11 10:32:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 10:32:52.563673 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fa106e-b83d-44c2-b70f-173fc37293af None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:52.572119 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a77d58c-bd07-4a96-96e0-cb9d81f5152c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:52.573017 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a77d58c-bd07-4a96-96e0-cb9d81f5152c None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:52.573479 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a77d58c-bd07-4a96-96e0-cb9d81f5152c None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:52.574476 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a77d58c-bd07-4a96-96e0-cb9d81f5152c None admin] PATH_INFO: `/v3/users/3570ee53f3c442848938999c912a848b` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:52.576453 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a77d58c-bd07-4a96-96e0-cb9d81f5152c None admin] RBAC: Authorizing `identity:get_user(user_id=3570ee53f3c442848938999c912a848b)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:52.578854 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a77d58c-bd07-4a96-96e0-cb9d81f5152c None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:52.581394 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 61/121] 10.222.0.15 () {56 vars in 1177 bytes} [Sun Sep 11 10:32:52 2022] GET /identity/v3/users/3570ee53f3c442848938999c912a848b => generated 247 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:32:52.582635 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a77d58c-bd07-4a96-96e0-cb9d81f5152c None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:52.664293 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:52.688381 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fa106e-b83d-44c2-b70f-173fc37293af None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:52.707315 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a77d58c-bd07-4a96-96e0-cb9d81f5152c None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:52.723675 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:52.734375 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119a9aee-3e1e-4ff3-94d7-a19492ddcc25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:52.735010 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a9aee-3e1e-4ff3-94d7-a19492ddcc25 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:52.735450 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a9aee-3e1e-4ff3-94d7-a19492ddcc25 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:52.735759 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a9aee-3e1e-4ff3-94d7-a19492ddcc25 None admin] PATH_INFO: `/v3/roles/member` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:52.748156 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95133217-5e38-4fc0-a5fe-8d8663d94b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:52.748982 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95133217-5e38-4fc0-a5fe-8d8663d94b2e None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:52.749150 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119a9aee-3e1e-4ff3-94d7-a19492ddcc25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:32:52.749608 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95133217-5e38-4fc0-a5fe-8d8663d94b2e None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:52.750029 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95133217-5e38-4fc0-a5fe-8d8663d94b2e None admin] PATH_INFO: `/v3/roles/member` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:52.754381 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-119a9aee-3e1e-4ff3-94d7-a19492ddcc25 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 10:32:52.754381 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:32:52.754381 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:32:52.754381 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:32:52.754381 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:32:52.754381 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:32:52.754381 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:32:52.754381 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:32:52.754381 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:32:52.754381 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:32:52.754381 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:32:52.754381 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:32:52.754381 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:32:52.754381 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:32:52.754381 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:32:52.754381 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:32:52.755413 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:32:52.755413 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:32:52.755413 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:32:52.755413 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:32:52.755413 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:32:52.755413 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:32:52.755413 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:32:52.755413 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:32:52.755413 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:32:52.755413 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:32:52.755413 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:32:52.755413 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:32:52.755413 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:32:52.755413 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:32:52.755413 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:32:52.755413 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:32:52.756490 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:32:52.756490 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:32:52.756490 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:32:52.756490 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:32:52.756490 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:32:52.756490 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:32:52.756490 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:32:52.756490 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:32:52.756490 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:32:52.756490 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:32:52.756490 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 10:32:52.756490 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:32:52.756490 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 61/122] 10.222.0.15 () {56 vars in 1099 bytes} [Sun Sep 11 10:32:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:32:52.756490 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119a9aee-3e1e-4ff3-94d7-a19492ddcc25 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:52.764831 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95133217-5e38-4fc0-a5fe-8d8663d94b2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:32:52.770818 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-95133217-5e38-4fc0-a5fe-8d8663d94b2e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 10:32:52.770818 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:32:52.770818 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:32:52.770818 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:32:52.770818 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:32:52.770818 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:32:52.770818 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:32:52.770818 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:32:52.770818 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:32:52.770818 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:32:52.770818 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:32:52.770818 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:32:52.770818 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:32:52.770818 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:32:52.770818 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:32:52.770818 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:32:52.771725 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:32:52.771725 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:32:52.771725 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:32:52.771725 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:32:52.771725 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:32:52.771725 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:32:52.771725 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:32:52.771725 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:32:52.771725 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:32:52.771725 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:32:52.771725 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:32:52.771725 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:32:52.771725 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:32:52.771725 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:32:52.771725 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:32:52.771725 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:32:52.772482 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:32:52.772482 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:32:52.772482 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:32:52.772482 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:32:52.772482 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:32:52.772482 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:32:52.772482 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:32:52.772482 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:32:52.772482 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:32:52.772482 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:32:52.772482 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 10:32:52.772482 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:32:52.775445 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 62/123] 10.222.0.15 () {56 vars in 1099 bytes} [Sun Sep 11 10:32:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:32:52.781360 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95133217-5e38-4fc0-a5fe-8d8663d94b2e None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:52.870913 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119a9aee-3e1e-4ff3-94d7-a19492ddcc25 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:52.886638 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95133217-5e38-4fc0-a5fe-8d8663d94b2e None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:52.909962 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1da89fc-c83c-4db0-a511-0a3abd415153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:52.910482 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:52.912143 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da89fc-c83c-4db0-a511-0a3abd415153 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:52.912565 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da89fc-c83c-4db0-a511-0a3abd415153 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:52.913164 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da89fc-c83c-4db0-a511-0a3abd415153 None admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:52.914734 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1da89fc-c83c-4db0-a511-0a3abd415153 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:32:52.915577 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1da89fc-c83c-4db0-a511-0a3abd415153 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:52.917992 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1da89fc-c83c-4db0-a511-0a3abd415153 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:52.921663 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b1280e-b5b0-4a06-b35b-1ee097ad92e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:52.922306 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b1280e-b5b0-4a06-b35b-1ee097ad92e6 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:52.922588 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b1280e-b5b0-4a06-b35b-1ee097ad92e6 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:52.922859 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b1280e-b5b0-4a06-b35b-1ee097ad92e6 None admin] PATH_INFO: `/v3/projects/1b12f8d03f4f4f4c8b5f37b804de18c5` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:52.924008 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b1280e-b5b0-4a06-b35b-1ee097ad92e6 None admin] RBAC: Authorizing `identity:get_project(project_id=1b12f8d03f4f4f4c8b5f37b804de18c5)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:52.925444 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b1280e-b5b0-4a06-b35b-1ee097ad92e6 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:52.927308 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 63/124] 10.222.0.15 () {56 vars in 1186 bytes} [Sun Sep 11 10:32:52 2022] GET /identity/v3/projects/1b12f8d03f4f4f4c8b5f37b804de18c5 => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:32:52.927904 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b1280e-b5b0-4a06-b35b-1ee097ad92e6 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:52.929052 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 62/125] 10.222.0.15 () {56 vars in 1111 bytes} [Sun Sep 11 10:32:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:32:52.930124 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1da89fc-c83c-4db0-a511-0a3abd415153 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:52.996529 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:53.056228 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1da89fc-c83c-4db0-a511-0a3abd415153 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:53.074625 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b1280e-b5b0-4a06-b35b-1ee097ad92e6 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:53.094883 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f69383-d3f8-4de9-b6e7-d5875793c07b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:53.095871 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f69383-d3f8-4de9-b6e7-d5875793c07b None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:53.096292 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f69383-d3f8-4de9-b6e7-d5875793c07b None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:53.096811 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f69383-d3f8-4de9-b6e7-d5875793c07b None admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:53.097607 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f69383-d3f8-4de9-b6e7-d5875793c07b None admin] RBAC: Adding query filter params (name=member) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:32:53.098066 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f69383-d3f8-4de9-b6e7-d5875793c07b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:53.099725 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f69383-d3f8-4de9-b6e7-d5875793c07b None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:53.109711 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 63/126] 10.222.0.15 () {56 vars in 1101 bytes} [Sun Sep 11 10:32:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:32:53.115171 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f69383-d3f8-4de9-b6e7-d5875793c07b None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:53.116186 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3244a5bb-a686-47d2-af40-1b9d12848772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:53.116913 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3244a5bb-a686-47d2-af40-1b9d12848772 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:53.117207 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3244a5bb-a686-47d2-af40-1b9d12848772 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:53.117486 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3244a5bb-a686-47d2-af40-1b9d12848772 None admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:53.118742 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3244a5bb-a686-47d2-af40-1b9d12848772 None admin] RBAC: Adding query filter params (name=member) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:32:53.119048 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3244a5bb-a686-47d2-af40-1b9d12848772 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:53.121673 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3244a5bb-a686-47d2-af40-1b9d12848772 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:53.132486 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 64/127] 10.222.0.15 () {56 vars in 1101 bytes} [Sun Sep 11 10:32:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:32:53.133507 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3244a5bb-a686-47d2-af40-1b9d12848772 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:53.213128 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:53.243865 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f69383-d3f8-4de9-b6e7-d5875793c07b None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:53.262535 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:53.266464 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3244a5bb-a686-47d2-af40-1b9d12848772 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:53.299958 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a7a912-265c-412e-aa4f-61e7ffbcedf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:53.300860 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a7a912-265c-412e-aa4f-61e7ffbcedf8 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:53.301147 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a7a912-265c-412e-aa4f-61e7ffbcedf8 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:53.301428 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a7a912-265c-412e-aa4f-61e7ffbcedf8 None admin] PATH_INFO: `/v3/users/0175a3d1c7824b08b476a3f905aad38e` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:53.303259 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a7a912-265c-412e-aa4f-61e7ffbcedf8 None admin] RBAC: Authorizing `identity:get_user(user_id=0175a3d1c7824b08b476a3f905aad38e)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:53.305437 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a7a912-265c-412e-aa4f-61e7ffbcedf8 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:53.309134 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 64/128] 10.222.0.15 () {56 vars in 1177 bytes} [Sun Sep 11 10:32:53 2022] GET /identity/v3/users/0175a3d1c7824b08b476a3f905aad38e => generated 275 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:32:53.310574 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a7a912-265c-412e-aa4f-61e7ffbcedf8 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:53.324202 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e60f9b1-f63e-492a-9a47-7b3b7bca22c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:53.325179 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e60f9b1-f63e-492a-9a47-7b3b7bca22c4 None admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:53.325662 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e60f9b1-f63e-492a-9a47-7b3b7bca22c4 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:53.326135 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e60f9b1-f63e-492a-9a47-7b3b7bca22c4 None admin] PATH_INFO: `/v3/projects/1b12f8d03f4f4f4c8b5f37b804de18c5/users/3570ee53f3c442848938999c912a848b/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:53.329531 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e60f9b1-f63e-492a-9a47-7b3b7bca22c4 None admin] RBAC: Authorizing `identity:create_grant(project_id=1b12f8d03f4f4f4c8b5f37b804de18c5, user_id=3570ee53f3c442848938999c912a848b, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:53.331988 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e60f9b1-f63e-492a-9a47-7b3b7bca22c4 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:53.351836 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 65/129] 10.222.0.15 () {58 vars in 1439 bytes} [Sun Sep 11 10:32:53 2022] PUT /identity/v3/projects/1b12f8d03f4f4f4c8b5f37b804de18c5/users/3570ee53f3c442848938999c912a848b/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:32:53.357194 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e60f9b1-f63e-492a-9a47-7b3b7bca22c4 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:53.507332 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a7a912-265c-412e-aa4f-61e7ffbcedf8 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:53.569931 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e60f9b1-f63e-492a-9a47-7b3b7bca22c4 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:53.583028 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7247205-bd83-4d16-a20f-1b55ad7e0231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:53.583701 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7247205-bd83-4d16-a20f-1b55ad7e0231 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:53.583979 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7247205-bd83-4d16-a20f-1b55ad7e0231 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:53.584249 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7247205-bd83-4d16-a20f-1b55ad7e0231 None admin] PATH_INFO: `/v3/users/0175a3d1c7824b08b476a3f905aad38e` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:53.585384 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7247205-bd83-4d16-a20f-1b55ad7e0231 None admin] RBAC: Authorizing `identity:get_user(user_id=0175a3d1c7824b08b476a3f905aad38e)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:53.587397 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7247205-bd83-4d16-a20f-1b55ad7e0231 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:53.589740 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 65/130] 10.222.0.15 () {56 vars in 1177 bytes} [Sun Sep 11 10:32:53 2022] GET /identity/v3/users/0175a3d1c7824b08b476a3f905aad38e => generated 275 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:32:53.592641 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7247205-bd83-4d16-a20f-1b55ad7e0231 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:53.593684 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:53.624524 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462c68b9-54e0-4892-983c-c130a9382767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:53.625832 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c68b9-54e0-4892-983c-c130a9382767 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:53.626313 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c68b9-54e0-4892-983c-c130a9382767 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:53.626957 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c68b9-54e0-4892-983c-c130a9382767 None admin] PATH_INFO: `/v3/users/0175a3d1c7824b08b476a3f905aad38e` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:53.629074 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462c68b9-54e0-4892-983c-c130a9382767 None admin] RBAC: Authorizing `identity:get_user(user_id=0175a3d1c7824b08b476a3f905aad38e)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:53.632380 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462c68b9-54e0-4892-983c-c130a9382767 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:53.633440 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 66/131] 10.222.0.15 () {56 vars in 1177 bytes} [Sun Sep 11 10:32:53 2022] GET /identity/v3/users/0175a3d1c7824b08b476a3f905aad38e => generated 275 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:32:53.634200 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462c68b9-54e0-4892-983c-c130a9382767 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:53.659395 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:53.731109 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7247205-bd83-4d16-a20f-1b55ad7e0231 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:53.760562 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462c68b9-54e0-4892-983c-c130a9382767 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:53.793117 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f4a7a3-a96b-4922-972f-924305b3ed4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:53.793796 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f4a7a3-a96b-4922-972f-924305b3ed4b None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:53.794092 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f4a7a3-a96b-4922-972f-924305b3ed4b None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:53.794417 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f4a7a3-a96b-4922-972f-924305b3ed4b None admin] PATH_INFO: `/v3/projects/1b12f8d03f4f4f4c8b5f37b804de18c5` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:53.798735 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f4a7a3-a96b-4922-972f-924305b3ed4b None admin] RBAC: Authorizing `identity:get_project(project_id=1b12f8d03f4f4f4c8b5f37b804de18c5)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:53.800219 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f4a7a3-a96b-4922-972f-924305b3ed4b None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:53.802350 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 66/132] 10.222.0.15 () {56 vars in 1186 bytes} [Sun Sep 11 10:32:53 2022] GET /identity/v3/projects/1b12f8d03f4f4f4c8b5f37b804de18c5 => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:32:53.804191 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f4a7a3-a96b-4922-972f-924305b3ed4b None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:53.829114 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19731b23-9839-48ff-a2db-cb886aa7ba93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:53.829725 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19731b23-9839-48ff-a2db-cb886aa7ba93 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:53.830010 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19731b23-9839-48ff-a2db-cb886aa7ba93 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:53.830286 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19731b23-9839-48ff-a2db-cb886aa7ba93 None admin] PATH_INFO: `/v3/projects/1b12f8d03f4f4f4c8b5f37b804de18c5` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:53.831688 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19731b23-9839-48ff-a2db-cb886aa7ba93 None admin] RBAC: Authorizing `identity:get_project(project_id=1b12f8d03f4f4f4c8b5f37b804de18c5)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:53.833868 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19731b23-9839-48ff-a2db-cb886aa7ba93 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:53.843384 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 67/133] 10.222.0.15 () {56 vars in 1186 bytes} [Sun Sep 11 10:32:53 2022] GET /identity/v3/projects/1b12f8d03f4f4f4c8b5f37b804de18c5 => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:32:53.844258 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19731b23-9839-48ff-a2db-cb886aa7ba93 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:53.963402 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:53.979871 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f4a7a3-a96b-4922-972f-924305b3ed4b None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:54.014934 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:54.037081 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c789e204-93f4-403e-8e0a-517f7485fa20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:54.037887 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c789e204-93f4-403e-8e0a-517f7485fa20 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:54.038175 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c789e204-93f4-403e-8e0a-517f7485fa20 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:54.038448 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c789e204-93f4-403e-8e0a-517f7485fa20 None admin] PATH_INFO: `/v3/projects/b21540a9dc144b76bf6e2e819b54cccf` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:54.039615 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c789e204-93f4-403e-8e0a-517f7485fa20 None admin] RBAC: Authorizing `identity:get_project(project_id=b21540a9dc144b76bf6e2e819b54cccf)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:54.041767 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c789e204-93f4-403e-8e0a-517f7485fa20 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:54.047749 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 67/134] 10.222.0.15 () {56 vars in 1186 bytes} [Sun Sep 11 10:32:53 2022] GET /identity/v3/projects/b21540a9dc144b76bf6e2e819b54cccf => generated 312 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:32:54.048665 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c789e204-93f4-403e-8e0a-517f7485fa20 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:54.066265 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19731b23-9839-48ff-a2db-cb886aa7ba93 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:54.105000 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3726aacf-dea6-41a5-9e25-b46f976e4f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:54.105806 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3726aacf-dea6-41a5-9e25-b46f976e4f5b None admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:54.106092 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3726aacf-dea6-41a5-9e25-b46f976e4f5b None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:54.106381 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3726aacf-dea6-41a5-9e25-b46f976e4f5b None admin] PATH_INFO: `/v3/projects/1b12f8d03f4f4f4c8b5f37b804de18c5/users/0175a3d1c7824b08b476a3f905aad38e/roles/cdb3aa1bd85b4694964629ff7cf8039b` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:54.108795 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3726aacf-dea6-41a5-9e25-b46f976e4f5b None admin] RBAC: Authorizing `identity:create_grant(project_id=1b12f8d03f4f4f4c8b5f37b804de18c5, user_id=0175a3d1c7824b08b476a3f905aad38e, role_id=cdb3aa1bd85b4694964629ff7cf8039b)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:54.110360 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3726aacf-dea6-41a5-9e25-b46f976e4f5b None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:54.128428 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 68/135] 10.222.0.15 () {58 vars in 1439 bytes} [Sun Sep 11 10:32:53 2022] PUT /identity/v3/projects/1b12f8d03f4f4f4c8b5f37b804de18c5/users/0175a3d1c7824b08b476a3f905aad38e/roles/cdb3aa1bd85b4694964629ff7cf8039b => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:32:54.130116 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79cd525-e4cd-4ef0-ad0b-7cfbbe55a307 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:54.130406 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79cd525-e4cd-4ef0-ad0b-7cfbbe55a307 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:54.130674 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79cd525-e4cd-4ef0-ad0b-7cfbbe55a307 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:54.131982 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 69/136] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:32:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:32:54.135527 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79cd525-e4cd-4ef0-ad0b-7cfbbe55a307 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:54.204513 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c789e204-93f4-403e-8e0a-517f7485fa20 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:54.284022 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09d358c-36f8-48e6-b759-4529fc052692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:54.286391 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09d358c-36f8-48e6-b759-4529fc052692 None admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:54.286994 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09d358c-36f8-48e6-b759-4529fc052692 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:54.287710 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09d358c-36f8-48e6-b759-4529fc052692 None admin] PATH_INFO: `/v3/projects/1b12f8d03f4f4f4c8b5f37b804de18c5/users/0175a3d1c7824b08b476a3f905aad38e/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:54.292703 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09d358c-36f8-48e6-b759-4529fc052692 None admin] RBAC: Authorizing `identity:create_grant(project_id=1b12f8d03f4f4f4c8b5f37b804de18c5, user_id=0175a3d1c7824b08b476a3f905aad38e, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:54.295154 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09d358c-36f8-48e6-b759-4529fc052692 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:54.329245 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 68/137] 10.222.0.15 () {58 vars in 1439 bytes} [Sun Sep 11 10:32:54 2022] PUT /identity/v3/projects/1b12f8d03f4f4f4c8b5f37b804de18c5/users/0175a3d1c7824b08b476a3f905aad38e/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:32:54.330995 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57141f6e-f622-46d8-8fb1-441538e5a584 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:54.331513 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57141f6e-f622-46d8-8fb1-441538e5a584 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:54.332018 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57141f6e-f622-46d8-8fb1-441538e5a584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:54.349751 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79cd525-e4cd-4ef0-ad0b-7cfbbe55a307 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:54.362926 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:54.376560 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-57141f6e-f622-46d8-8fb1-441538e5a584 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:32:54.377784 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-57141f6e-f622-46d8-8fb1-441538e5a584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:32:54.410409 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6230c77f-4233-4f70-8680-4a688e692d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:54.411090 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230c77f-4233-4f70-8680-4a688e692d44 None admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:54.411371 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230c77f-4233-4f70-8680-4a688e692d44 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:54.411666 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230c77f-4233-4f70-8680-4a688e692d44 None admin] PATH_INFO: `/v3/projects/b21540a9dc144b76bf6e2e819b54cccf/users/0175a3d1c7824b08b476a3f905aad38e/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:54.416806 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6230c77f-4233-4f70-8680-4a688e692d44 None admin] RBAC: Authorizing `identity:create_grant(project_id=b21540a9dc144b76bf6e2e819b54cccf, user_id=0175a3d1c7824b08b476a3f905aad38e, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:54.418286 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6230c77f-4233-4f70-8680-4a688e692d44 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:54.447732 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 70/138] 10.222.0.15 () {58 vars in 1439 bytes} [Sun Sep 11 10:32:54 2022] PUT /identity/v3/projects/b21540a9dc144b76bf6e2e819b54cccf/users/0175a3d1c7824b08b476a3f905aad38e/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:32:54.449313 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c6aa90-0552-4085-9049-5c382e23d9f9 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:54.449605 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c6aa90-0552-4085-9049-5c382e23d9f9 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:54.449886 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c6aa90-0552-4085-9049-5c382e23d9f9 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:54.451340 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 71/139] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:32:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:32:54.461170 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c11e563-e1d8-4990-8c84-834020517671 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:54.461466 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c11e563-e1d8-4990-8c84-834020517671 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:54.461721 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c11e563-e1d8-4990-8c84-834020517671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:54.471498 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:54.502525 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-5c11e563-e1d8-4990-8c84-834020517671 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:32:54.503814 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-5c11e563-e1d8-4990-8c84-834020517671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:32:54.547412 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 69/140] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:32:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:32:54.637097 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 72/141] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:32:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:32:54.897942 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57141f6e-f622-46d8-8fb1-441538e5a584 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:54.933665 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c11e563-e1d8-4990-8c84-834020517671 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:55.007540 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57141f6e-f622-46d8-8fb1-441538e5a584 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:55.047675 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d50b286-2e84-44df-8a32-f6d5b5da8b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:55.048320 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d50b286-2e84-44df-8a32-f6d5b5da8b7d None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:55.048606 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d50b286-2e84-44df-8a32-f6d5b5da8b7d None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:55.048879 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d50b286-2e84-44df-8a32-f6d5b5da8b7d None admin] PATH_INFO: `/v3/domains/default` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:55.050000 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d50b286-2e84-44df-8a32-f6d5b5da8b7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:55.051479 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d50b286-2e84-44df-8a32-f6d5b5da8b7d None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:55.053403 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 70/142] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:32:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:32:55.060215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d50b286-2e84-44df-8a32-f6d5b5da8b7d None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:55.108509 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:55.135816 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:55.140009 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c11e563-e1d8-4990-8c84-834020517671 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:55.173012 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d50b286-2e84-44df-8a32-f6d5b5da8b7d None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:55.206128 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68efbee4-ff50-49e7-ba1d-d4f3dc5f7ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:55.206796 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68efbee4-ff50-49e7-ba1d-d4f3dc5f7ee4 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:55.207128 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68efbee4-ff50-49e7-ba1d-d4f3dc5f7ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:55.207546 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68efbee4-ff50-49e7-ba1d-d4f3dc5f7ee4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:55.216192 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1956680-7ca0-4536-b84f-20962d841e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:55.216813 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1956680-7ca0-4536-b84f-20962d841e0a None admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:55.217092 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1956680-7ca0-4536-b84f-20962d841e0a None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:55.217361 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1956680-7ca0-4536-b84f-20962d841e0a None admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:55.218018 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1956680-7ca0-4536-b84f-20962d841e0a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:55.219709 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1956680-7ca0-4536-b84f-20962d841e0a None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:55.234505 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68efbee4-ff50-49e7-ba1d-d4f3dc5f7ee4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:32:55.239320 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-68efbee4-ff50-49e7-ba1d-d4f3dc5f7ee4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 10:32:55.239320 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:32:55.239320 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:32:55.239320 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:32:55.239320 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:32:55.239320 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:32:55.239320 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:32:55.239320 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:32:55.239320 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:32:55.239320 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:32:55.239320 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:32:55.239320 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:32:55.239320 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:32:55.239320 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:32:55.239320 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:32:55.239320 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:32:55.240233 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:32:55.240233 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:32:55.240233 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:32:55.240233 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:32:55.240233 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:32:55.240233 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:32:55.240233 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:32:55.240233 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:32:55.240233 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:32:55.240233 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:32:55.240233 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:32:55.240233 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:32:55.240233 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:32:55.240233 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:32:55.240233 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:32:55.240233 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:32:55.241038 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:32:55.241038 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:32:55.241038 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:32:55.241038 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:32:55.241038 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:32:55.241038 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:32:55.241038 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:32:55.241038 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:32:55.241038 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:32:55.241038 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:32:55.241038 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 10:32:55.241038 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:32:55.241038 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 73/143] 10.222.0.15 () {56 vars in 1099 bytes} [Sun Sep 11 10:32:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 10:32:55.252615 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68efbee4-ff50-49e7-ba1d-d4f3dc5f7ee4 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:55.266886 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 71/144] 10.222.0.15 () {60 vars in 1132 bytes} [Sun Sep 11 10:32:55 2022] POST /identity/v3/users => generated 283 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 10:32:55.406466 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68efbee4-ff50-49e7-ba1d-d4f3dc5f7ee4 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:55.444700 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6d392f-8d16-4dd3-8e0c-0bef64478342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:55.445232 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6d392f-8d16-4dd3-8e0c-0bef64478342 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:55.445506 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6d392f-8d16-4dd3-8e0c-0bef64478342 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:55.445831 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6d392f-8d16-4dd3-8e0c-0bef64478342 None admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:55.447536 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6d392f-8d16-4dd3-8e0c-0bef64478342 None admin] RBAC: Adding query filter params (name=reader) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:32:55.448617 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6d392f-8d16-4dd3-8e0c-0bef64478342 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:55.450050 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6d392f-8d16-4dd3-8e0c-0bef64478342 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:55.459312 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 74/145] 10.222.0.15 () {56 vars in 1101 bytes} [Sun Sep 11 10:32:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:32:55.466266 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1956680-7ca0-4536-b84f-20962d841e0a None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:55.591363 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:55.595258 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1956680-7ca0-4536-b84f-20962d841e0a None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:55.636057 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b43549-ad0e-498b-9cec-31d4f1c256e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:55.636693 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b43549-ad0e-498b-9cec-31d4f1c256e0 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:55.636972 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b43549-ad0e-498b-9cec-31d4f1c256e0 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:55.637239 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b43549-ad0e-498b-9cec-31d4f1c256e0 None admin] PATH_INFO: `/v3/users/53e4b2016de34e7bb3adfa8fa78a1610` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:55.638374 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b43549-ad0e-498b-9cec-31d4f1c256e0 None admin] RBAC: Authorizing `identity:get_user(user_id=53e4b2016de34e7bb3adfa8fa78a1610)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:55.639846 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b43549-ad0e-498b-9cec-31d4f1c256e0 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:55.641769 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 72/146] 10.222.0.15 () {56 vars in 1177 bytes} [Sun Sep 11 10:32:55 2022] GET /identity/v3/users/53e4b2016de34e7bb3adfa8fa78a1610 => generated 289 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:32:55.649864 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6d392f-8d16-4dd3-8e0c-0bef64478342 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:55.688512 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:55.787882 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6d392f-8d16-4dd3-8e0c-0bef64478342 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:55.826596 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa54d422-bb50-4e0f-9bbc-3c652574f189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:55.827237 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa54d422-bb50-4e0f-9bbc-3c652574f189 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:55.827543 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa54d422-bb50-4e0f-9bbc-3c652574f189 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:55.827817 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa54d422-bb50-4e0f-9bbc-3c652574f189 None admin] PATH_INFO: `/v3/projects/1b12f8d03f4f4f4c8b5f37b804de18c5` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:55.828938 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa54d422-bb50-4e0f-9bbc-3c652574f189 None admin] RBAC: Authorizing `identity:get_project(project_id=1b12f8d03f4f4f4c8b5f37b804de18c5)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:55.830439 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa54d422-bb50-4e0f-9bbc-3c652574f189 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:55.832130 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 75/147] 10.222.0.15 () {56 vars in 1186 bytes} [Sun Sep 11 10:32:55 2022] GET /identity/v3/projects/1b12f8d03f4f4f4c8b5f37b804de18c5 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:32:55.838319 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b43549-ad0e-498b-9cec-31d4f1c256e0 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:55.952207 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b43549-ad0e-498b-9cec-31d4f1c256e0 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:55.990370 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1137a609-b8d1-4011-9d2b-870da3e7c73a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:55.991029 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1137a609-b8d1-4011-9d2b-870da3e7c73a None admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:55.991309 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1137a609-b8d1-4011-9d2b-870da3e7c73a None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:55.991656 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1137a609-b8d1-4011-9d2b-870da3e7c73a None admin] PATH_INFO: `/v3/projects/1b12f8d03f4f4f4c8b5f37b804de18c5/users/53e4b2016de34e7bb3adfa8fa78a1610/roles/d577f4ba2f9c412e9530e50d7f3d6bd7` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:55.994015 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1137a609-b8d1-4011-9d2b-870da3e7c73a None admin] RBAC: Authorizing `identity:create_grant(project_id=1b12f8d03f4f4f4c8b5f37b804de18c5, user_id=53e4b2016de34e7bb3adfa8fa78a1610, role_id=d577f4ba2f9c412e9530e50d7f3d6bd7)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:55.995480 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1137a609-b8d1-4011-9d2b-870da3e7c73a None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:56.016947 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 73/148] 10.222.0.15 () {58 vars in 1439 bytes} [Sun Sep 11 10:32:55 2022] PUT /identity/v3/projects/1b12f8d03f4f4f4c8b5f37b804de18c5/users/53e4b2016de34e7bb3adfa8fa78a1610/roles/d577f4ba2f9c412e9530e50d7f3d6bd7 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:32:57.158060 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28086f8-7a20-4781-aac5-e46da056c289 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:57.158789 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28086f8-7a20-4781-aac5-e46da056c289 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:57.158789 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28086f8-7a20-4781-aac5-e46da056c289 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:57.159997 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 76/149] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:32:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:32:57.166410 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b36e9e-70af-45eb-a945-dc113b93147b None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:57.167116 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b36e9e-70af-45eb-a945-dc113b93147b None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:57.167116 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b36e9e-70af-45eb-a945-dc113b93147b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:57.199021 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-d5b36e9e-70af-45eb-a945-dc113b93147b None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:32:57.200272 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-d5b36e9e-70af-45eb-a945-dc113b93147b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:32:57.236754 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:57.307791 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 74/150] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:32:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:32:57.427640 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81332186-4773-401e-8c94-c304e6329564 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:57.428082 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81332186-4773-401e-8c94-c304e6329564 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:57.428595 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81332186-4773-401e-8c94-c304e6329564 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:57.431271 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 77/151] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:32:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:32:57.437984 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab87bae-5441-4c5d-a190-7ff9bb0ef0ad None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:57.438590 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab87bae-5441-4c5d-a190-7ff9bb0ef0ad None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:57.439067 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab87bae-5441-4c5d-a190-7ff9bb0ef0ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:57.468466 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-8ab87bae-5441-4c5d-a190-7ff9bb0ef0ad None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:32:57.469661 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-8ab87bae-5441-4c5d-a190-7ff9bb0ef0ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:32:57.559519 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0026bade-dfc1-40bd-9d35-1b7a1fe4338f None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:57.560196 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0026bade-dfc1-40bd-9d35-1b7a1fe4338f None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:57.560699 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0026bade-dfc1-40bd-9d35-1b7a1fe4338f None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:57.563388 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 78/152] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:32:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:32:57.567367 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d37afa6-f63a-4932-b167-61e8d83e907a None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:57.567925 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d37afa6-f63a-4932-b167-61e8d83e907a None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:57.568408 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d37afa6-f63a-4932-b167-61e8d83e907a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:57.568614 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 75/153] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:32:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:32:57.608437 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-6d37afa6-f63a-4932-b167-61e8d83e907a None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:32:57.610079 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-6d37afa6-f63a-4932-b167-61e8d83e907a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:32:57.714660 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 79/154] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:32:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 10:32:57.763802 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab87bae-5441-4c5d-a190-7ff9bb0ef0ad None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:57.878518 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:57.899278 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab87bae-5441-4c5d-a190-7ff9bb0ef0ad None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:57.941711 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6ac1cc-8572-49b4-8e3b-fa114a0d8892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:57.942358 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ac1cc-8572-49b4-8e3b-fa114a0d8892 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:57.942644 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ac1cc-8572-49b4-8e3b-fa114a0d8892 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:57.942914 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ac1cc-8572-49b4-8e3b-fa114a0d8892 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:57.960459 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6ac1cc-8572-49b4-8e3b-fa114a0d8892 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:32:57.965823 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-5b6ac1cc-8572-49b4-8e3b-fa114a0d8892 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 10:32:57.965823 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:32:57.965823 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:32:57.965823 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:32:57.965823 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:32:57.965823 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:32:57.965823 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:32:57.965823 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:32:57.965823 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:32:57.965823 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:32:57.965823 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:32:57.965823 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:32:57.965823 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:32:57.965823 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:32:57.965823 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:32:57.965823 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:32:57.966690 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:32:57.966690 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:32:57.966690 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:32:57.966690 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:32:57.966690 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:32:57.966690 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:32:57.966690 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:32:57.966690 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:32:57.966690 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:32:57.966690 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:32:57.966690 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:32:57.966690 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:32:57.966690 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:32:57.966690 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:32:57.966690 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:32:57.966690 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:32:57.967911 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:32:57.967911 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:32:57.967911 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:32:57.967911 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:32:57.967911 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:32:57.967911 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:32:57.967911 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:32:57.967911 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:32:57.967911 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:32:57.967911 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:32:57.967911 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 10:32:57.967911 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:32:57.967911 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 76/155] 10.222.0.15 () {56 vars in 1114 bytes} [Sun Sep 11 10:32:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:32:57.974689 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d37afa6-f63a-4932-b167-61e8d83e907a None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:58.016052 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:58.121923 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d37afa6-f63a-4932-b167-61e8d83e907a None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:58.169937 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d5b09f-6852-4a4d-9491-d5fdab7b0016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:58.170572 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d5b09f-6852-4a4d-9491-d5fdab7b0016 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:58.170837 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d5b09f-6852-4a4d-9491-d5fdab7b0016 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:58.171092 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d5b09f-6852-4a4d-9491-d5fdab7b0016 None admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:58.171788 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d5b09f-6852-4a4d-9491-d5fdab7b0016 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:32:58.172089 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d5b09f-6852-4a4d-9491-d5fdab7b0016 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:58.173463 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d5b09f-6852-4a4d-9491-d5fdab7b0016 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:58.186088 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 80/156] 10.222.0.15 () {56 vars in 1111 bytes} [Sun Sep 11 10:32:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:32:58.192242 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6ac1cc-8572-49b4-8e3b-fa114a0d8892 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:58.298587 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d5b09f-6852-4a4d-9491-d5fdab7b0016 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:58.438054 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6ac1cc-8572-49b4-8e3b-fa114a0d8892 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:58.478650 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:58.528256 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07d0d41-5137-4039-84c2-6d68a87cd8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:58.528908 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07d0d41-5137-4039-84c2-6d68a87cd8d3 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:58.529217 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07d0d41-5137-4039-84c2-6d68a87cd8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:58.529457 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07d0d41-5137-4039-84c2-6d68a87cd8d3 None admin] PATH_INFO: `/v3/users/0175a3d1c7824b08b476a3f905aad38e` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:58.530606 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07d0d41-5137-4039-84c2-6d68a87cd8d3 None admin] RBAC: Authorizing `identity:get_user(user_id=0175a3d1c7824b08b476a3f905aad38e)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:58.531056 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d5b09f-6852-4a4d-9491-d5fdab7b0016 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:58.532114 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07d0d41-5137-4039-84c2-6d68a87cd8d3 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:58.533976 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 77/157] 10.222.0.15 () {56 vars in 1177 bytes} [Sun Sep 11 10:32:58 2022] GET /identity/v3/users/0175a3d1c7824b08b476a3f905aad38e => generated 275 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:32:58.534750 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07d0d41-5137-4039-84c2-6d68a87cd8d3 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:58.569414 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:58.632687 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6665a16b-ca09-43f8-81be-048f37b7947a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:58.633336 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6665a16b-ca09-43f8-81be-048f37b7947a None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:58.634150 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6665a16b-ca09-43f8-81be-048f37b7947a None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:58.634424 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6665a16b-ca09-43f8-81be-048f37b7947a None admin] PATH_INFO: `/v3/roles/member` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:58.672383 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6665a16b-ca09-43f8-81be-048f37b7947a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:32:58.677727 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-6665a16b-ca09-43f8-81be-048f37b7947a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 10:32:58.677727 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:32:58.677727 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:32:58.677727 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:32:58.677727 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:32:58.677727 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:32:58.677727 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:32:58.677727 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:32:58.677727 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:32:58.677727 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:32:58.677727 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:32:58.677727 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:32:58.677727 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:32:58.677727 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:32:58.677727 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:32:58.677727 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:32:58.678680 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:32:58.678680 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:32:58.678680 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:32:58.678680 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:32:58.678680 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:32:58.678680 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:32:58.678680 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:32:58.678680 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:32:58.678680 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:32:58.678680 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:32:58.678680 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:32:58.678680 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:32:58.678680 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:32:58.678680 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:32:58.678680 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:32:58.678680 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:32:58.679494 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:32:58.679494 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:32:58.679494 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:32:58.679494 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:32:58.679494 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:32:58.679494 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:32:58.679494 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:32:58.679494 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:32:58.679494 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:32:58.679494 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:32:58.679494 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 10:32:58.679494 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:32:58.683813 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 81/158] 10.222.0.15 () {56 vars in 1099 bytes} [Sun Sep 11 10:32:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 386 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:32:58.685033 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a900398-b9c3-4707-a7e0-e73b15b53edb None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:58.685320 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a900398-b9c3-4707-a7e0-e73b15b53edb None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:58.685594 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a900398-b9c3-4707-a7e0-e73b15b53edb None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:58.687116 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 82/159] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:32:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:32:58.687846 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a900398-b9c3-4707-a7e0-e73b15b53edb None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:58.757541 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07d0d41-5137-4039-84c2-6d68a87cd8d3 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:58.829055 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e220cd6-1a6a-42fb-893e-7c1101defa80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:58.829679 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e220cd6-1a6a-42fb-893e-7c1101defa80 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:58.829963 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e220cd6-1a6a-42fb-893e-7c1101defa80 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:58.830237 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e220cd6-1a6a-42fb-893e-7c1101defa80 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:58.856451 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e220cd6-1a6a-42fb-893e-7c1101defa80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:32:58.861464 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-5e220cd6-1a6a-42fb-893e-7c1101defa80 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 10:32:58.861464 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:32:58.861464 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:32:58.861464 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:32:58.861464 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:32:58.861464 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:32:58.861464 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:32:58.861464 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:32:58.861464 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:32:58.861464 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:32:58.861464 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:32:58.861464 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:32:58.861464 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:32:58.861464 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:32:58.861464 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:32:58.861464 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:32:58.862392 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:32:58.862392 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:32:58.862392 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:32:58.862392 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:32:58.862392 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:32:58.862392 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:32:58.862392 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:32:58.862392 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:32:58.862392 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:32:58.862392 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:32:58.862392 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:32:58.862392 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:32:58.862392 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:32:58.862392 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:32:58.862392 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:32:58.862392 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:32:58.863709 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:32:58.863709 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:32:58.863709 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:32:58.863709 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:32:58.863709 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:32:58.863709 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:32:58.863709 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:32:58.863709 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:32:58.863709 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:32:58.863709 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:32:58.863709 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 10:32:58.863709 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:32:58.863709 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 78/160] 10.222.0.15 () {56 vars in 1096 bytes} [Sun Sep 11 10:32:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:32:58.863709 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e220cd6-1a6a-42fb-893e-7c1101defa80 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:58.944117 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a900398-b9c3-4707-a7e0-e73b15b53edb None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:58.995885 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:59.006468 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e9ee4a-3301-4963-b361-0cf7649cdf8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:59.007102 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e9ee4a-3301-4963-b361-0cf7649cdf8c None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:59.007380 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e9ee4a-3301-4963-b361-0cf7649cdf8c None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:59.007673 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e9ee4a-3301-4963-b361-0cf7649cdf8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:59.027368 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:59.061695 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e220cd6-1a6a-42fb-893e-7c1101defa80 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:59.129299 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fc19f9-e1a3-4df7-9b27-856164c2de5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:59.129843 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc19f9-e1a3-4df7-9b27-856164c2de5c None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:59.130124 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc19f9-e1a3-4df7-9b27-856164c2de5c None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:59.130397 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc19f9-e1a3-4df7-9b27-856164c2de5c None admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:59.131228 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fc19f9-e1a3-4df7-9b27-856164c2de5c None admin] RBAC: Adding query filter params (name=member) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:32:59.131559 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fc19f9-e1a3-4df7-9b27-856164c2de5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:59.133301 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fc19f9-e1a3-4df7-9b27-856164c2de5c None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:59.153843 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 79/161] 10.222.0.15 () {56 vars in 1101 bytes} [Sun Sep 11 10:32:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:32:59.155062 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2041c91c-3fa0-42a5-8db8-268fc98ce86f None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:59.155344 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2041c91c-3fa0-42a5-8db8-268fc98ce86f None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:59.160571 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e9ee4a-3301-4963-b361-0cf7649cdf8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:59.162060 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e9ee4a-3301-4963-b361-0cf7649cdf8c None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:59.164047 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2041c91c-3fa0-42a5-8db8-268fc98ce86f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:59.206126 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-2041c91c-3fa0-42a5-8db8-268fc98ce86f None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:32:59.207337 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-2041c91c-3fa0-42a5-8db8-268fc98ce86f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:32:59.320326 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 80/162] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:32:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:32:59.328513 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2041c91c-3fa0-42a5-8db8-268fc98ce86f None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:59.384174 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 83/163] 10.222.0.15 () {58 vars in 1282 bytes} [Sun Sep 11 10:32:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 691 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 10:32:59.385793 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e9ee4a-3301-4963-b361-0cf7649cdf8c None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:59.458675 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:59.464352 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:59.525789 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2041c91c-3fa0-42a5-8db8-268fc98ce86f None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:59.562670 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e9ee4a-3301-4963-b361-0cf7649cdf8c None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:59.582002 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97642bb-2700-443f-bd85-c3fb0b8e4680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:59.582790 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97642bb-2700-443f-bd85-c3fb0b8e4680 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:59.583278 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97642bb-2700-443f-bd85-c3fb0b8e4680 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:59.583674 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97642bb-2700-443f-bd85-c3fb0b8e4680 None admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:59.584467 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97642bb-2700-443f-bd85-c3fb0b8e4680 None admin] RBAC: Adding query filter params (name=admin) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:32:59.584873 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97642bb-2700-443f-bd85-c3fb0b8e4680 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:59.588349 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97642bb-2700-443f-bd85-c3fb0b8e4680 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:59.607052 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 81/164] 10.222.0.15 () {56 vars in 1099 bytes} [Sun Sep 11 10:32:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:32:59.607052 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97642bb-2700-443f-bd85-c3fb0b8e4680 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:59.631357 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54c395e-4026-4f94-b8ef-ed00525c845a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:59.632432 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54c395e-4026-4f94-b8ef-ed00525c845a None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:59.632865 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54c395e-4026-4f94-b8ef-ed00525c845a None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:59.633281 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54c395e-4026-4f94-b8ef-ed00525c845a None admin] PATH_INFO: `/v3/users/0175a3d1c7824b08b476a3f905aad38e` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:59.638169 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54c395e-4026-4f94-b8ef-ed00525c845a None admin] RBAC: Authorizing `identity:get_user(user_id=0175a3d1c7824b08b476a3f905aad38e)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:59.639907 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54c395e-4026-4f94-b8ef-ed00525c845a None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:59.642085 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 84/165] 10.222.0.15 () {56 vars in 1177 bytes} [Sun Sep 11 10:32:59 2022] GET /identity/v3/users/0175a3d1c7824b08b476a3f905aad38e => generated 275 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:32:59.642988 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54c395e-4026-4f94-b8ef-ed00525c845a None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:59.766114 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97642bb-2700-443f-bd85-c3fb0b8e4680 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:59.806279 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:59.814299 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54c395e-4026-4f94-b8ef-ed00525c845a None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:32:59.823683 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4f5817-7ae8-40d7-9c33-e159543e2d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:59.824424 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f5817-7ae8-40d7-9c33-e159543e2d76 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:59.824793 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f5817-7ae8-40d7-9c33-e159543e2d76 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:59.825137 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f5817-7ae8-40d7-9c33-e159543e2d76 None admin] PATH_INFO: `/v3/projects/1b12f8d03f4f4f4c8b5f37b804de18c5` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:59.829197 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4f5817-7ae8-40d7-9c33-e159543e2d76 None admin] RBAC: Authorizing `identity:get_project(project_id=1b12f8d03f4f4f4c8b5f37b804de18c5)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:59.829197 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4f5817-7ae8-40d7-9c33-e159543e2d76 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:59.830783 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 82/166] 10.222.0.15 () {56 vars in 1186 bytes} [Sun Sep 11 10:32:59 2022] GET /identity/v3/projects/1b12f8d03f4f4f4c8b5f37b804de18c5 => generated 298 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:32:59.835826 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:32:59.837971 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4f5817-7ae8-40d7-9c33-e159543e2d76 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:32:59.873306 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07050fa-1f4f-4c9d-b6d7-7834b37d5757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:32:59.874329 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07050fa-1f4f-4c9d-b6d7-7834b37d5757 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:32:59.874804 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07050fa-1f4f-4c9d-b6d7-7834b37d5757 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:32:59.875295 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07050fa-1f4f-4c9d-b6d7-7834b37d5757 None admin] PATH_INFO: `/v3/users/3570ee53f3c442848938999c912a848b` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:32:59.879661 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07050fa-1f4f-4c9d-b6d7-7834b37d5757 None admin] RBAC: Authorizing `identity:get_user(user_id=3570ee53f3c442848938999c912a848b)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:32:59.881500 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07050fa-1f4f-4c9d-b6d7-7834b37d5757 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:32:59.883529 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 85/167] 10.222.0.15 () {56 vars in 1177 bytes} [Sun Sep 11 10:32:59 2022] GET /identity/v3/users/3570ee53f3c442848938999c912a848b => generated 247 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:32:59.884712 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07050fa-1f4f-4c9d-b6d7-7834b37d5757 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:00.023738 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4f5817-7ae8-40d7-9c33-e159543e2d76 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:00.048271 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07050fa-1f4f-4c9d-b6d7-7834b37d5757 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:00.087166 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f92214-3ad9-4fd8-bc6e-862da6bf4bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:00.087809 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f92214-3ad9-4fd8-bc6e-862da6bf4bff None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:00.088091 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f92214-3ad9-4fd8-bc6e-862da6bf4bff None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:00.088363 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f92214-3ad9-4fd8-bc6e-862da6bf4bff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:00.120463 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7874086-456f-4bb5-9cde-3077f639437c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:00.121089 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7874086-456f-4bb5-9cde-3077f639437c None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:00.121370 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7874086-456f-4bb5-9cde-3077f639437c None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:00.121650 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7874086-456f-4bb5-9cde-3077f639437c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:00.122384 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7874086-456f-4bb5-9cde-3077f639437c None admin] RBAC: Adding query filter params (role.id=cdb3aa1bd85b4694964629ff7cf8039b, scope.project.id=1b12f8d03f4f4f4c8b5f37b804de18c5, user.id=0175a3d1c7824b08b476a3f905aad38e) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:00.122694 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7874086-456f-4bb5-9cde-3077f639437c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:00.124274 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7874086-456f-4bb5-9cde-3077f639437c None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:00.145706 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 86/168] 10.222.0.15 () {56 vars in 1374 bytes} [Sun Sep 11 10:32:59 2022] GET /identity/v3/role_assignments?user.id=0175a3d1c7824b08b476a3f905aad38e&scope.project.id=1b12f8d03f4f4f4c8b5f37b804de18c5&role.id=cdb3aa1bd85b4694964629ff7cf8039b => generated 608 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:00.146581 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7874086-456f-4bb5-9cde-3077f639437c None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:00.189125 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:00.201956 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f92214-3ad9-4fd8-bc6e-862da6bf4bff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:00.203453 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f92214-3ad9-4fd8-bc6e-862da6bf4bff None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:00.260196 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:00.411613 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7874086-456f-4bb5-9cde-3077f639437c None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:00.418315 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 83/169] 10.222.0.15 () {58 vars in 1282 bytes} [Sun Sep 11 10:32:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 587 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 10:33:00.418977 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f92214-3ad9-4fd8-bc6e-862da6bf4bff None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:00.474229 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8a7ca6-04d2-4177-9ae8-54baa5ad9766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:00.474874 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8a7ca6-04d2-4177-9ae8-54baa5ad9766 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:00.475156 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8a7ca6-04d2-4177-9ae8-54baa5ad9766 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:00.475450 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8a7ca6-04d2-4177-9ae8-54baa5ad9766 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:00.626197 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:00.626361 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8a7ca6-04d2-4177-9ae8-54baa5ad9766 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:00.627849 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8a7ca6-04d2-4177-9ae8-54baa5ad9766 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:00.633120 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f92214-3ad9-4fd8-bc6e-862da6bf4bff None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:00.676856 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:00.680633 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43270b96-8acb-4298-b0b6-4125efcfa343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:00.681270 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43270b96-8acb-4298-b0b6-4125efcfa343 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:00.681548 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43270b96-8acb-4298-b0b6-4125efcfa343 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:00.681818 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43270b96-8acb-4298-b0b6-4125efcfa343 None admin] PATH_INFO: `/v3/roles/member` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:00.692077 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43270b96-8acb-4298-b0b6-4125efcfa343 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:33:00.697569 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-43270b96-8acb-4298-b0b6-4125efcfa343 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 10:33:00.697569 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:00.697569 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:00.697569 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:00.697569 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:00.697569 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:00.697569 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:00.697569 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:00.697569 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:00.697569 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:00.697569 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:00.697569 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:00.697569 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:33:00.697569 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:33:00.697569 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:33:00.697569 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:33:00.698497 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:33:00.698497 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:33:00.698497 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:33:00.698497 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:33:00.698497 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:00.698497 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:00.698497 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:00.698497 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:00.698497 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:00.698497 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:00.698497 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:00.698497 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:00.698497 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:00.698497 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:00.698497 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:00.698497 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:00.699385 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:00.699385 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:00.699385 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:00.699385 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:00.699385 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:33:00.699385 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:33:00.699385 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:33:00.699385 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:33:00.699385 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:33:00.699385 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:33:00.699385 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 10:33:00.699385 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:33:00.699385 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 84/170] 10.222.0.15 () {56 vars in 1099 bytes} [Sun Sep 11 10:33:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:00.700088 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ddd571-bf7c-42aa-9531-8e85312e38a0 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:00.700155 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ddd571-bf7c-42aa-9531-8e85312e38a0 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:00.700399 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ddd571-bf7c-42aa-9531-8e85312e38a0 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:00.702089 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 85/171] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:33:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:33:00.704365 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ddd571-bf7c-42aa-9531-8e85312e38a0 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:00.738632 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 87/172] 10.222.0.15 () {58 vars in 1282 bytes} [Sun Sep 11 10:33:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 592 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 10:33:00.739994 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d95c20-cebb-496a-9340-840b76351557 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:00.740284 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d95c20-cebb-496a-9340-840b76351557 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:00.740565 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d95c20-cebb-496a-9340-840b76351557 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:00.741948 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 88/173] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:33:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:33:00.742890 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26afaa7-6d77-4f34-b3a4-070854bb510c None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:00.743362 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26afaa7-6d77-4f34-b3a4-070854bb510c None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:00.743694 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26afaa7-6d77-4f34-b3a4-070854bb510c None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:00.744965 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 89/174] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:33:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:33:00.747860 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b9da3-b10f-42da-abe2-e00098b6b546 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:00.748148 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b9da3-b10f-42da-abe2-e00098b6b546 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:00.748773 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b9da3-b10f-42da-abe2-e00098b6b546 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:00.750042 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 90/175] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:33:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:33:00.750742 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361b9da3-b10f-42da-abe2-e00098b6b546 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:00.809558 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ddd571-bf7c-42aa-9531-8e85312e38a0 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:00.845365 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ca5a8d-1c96-4874-a140-a9875538fe84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:00.845985 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca5a8d-1c96-4874-a140-a9875538fe84 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:00.846256 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca5a8d-1c96-4874-a140-a9875538fe84 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:00.846527 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca5a8d-1c96-4874-a140-a9875538fe84 None admin] PATH_INFO: `/v3/projects/1b12f8d03f4f4f4c8b5f37b804de18c5` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:00.847650 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ca5a8d-1c96-4874-a140-a9875538fe84 None admin] RBAC: Authorizing `identity:get_project(project_id=1b12f8d03f4f4f4c8b5f37b804de18c5)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:00.849090 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ca5a8d-1c96-4874-a140-a9875538fe84 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:00.850600 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361b9da3-b10f-42da-abe2-e00098b6b546 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:00.850715 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 86/176] 10.222.0.15 () {56 vars in 1186 bytes} [Sun Sep 11 10:33:00 2022] GET /identity/v3/projects/1b12f8d03f4f4f4c8b5f37b804de18c5 => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:00.852088 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad57f9d6-c830-4c6c-b9ba-602fc34d8528 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:00.853015 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad57f9d6-c830-4c6c-b9ba-602fc34d8528 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:00.853476 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad57f9d6-c830-4c6c-b9ba-602fc34d8528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:00.872049 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-ad57f9d6-c830-4c6c-b9ba-602fc34d8528 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:33:00.873259 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-ad57f9d6-c830-4c6c-b9ba-602fc34d8528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:33:00.885239 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662e9602-467e-4643-823d-d4b9186a4736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:00.885791 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e9602-467e-4643-823d-d4b9186a4736 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:00.886067 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e9602-467e-4643-823d-d4b9186a4736 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:00.886337 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e9602-467e-4643-823d-d4b9186a4736 None admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:00.886991 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662e9602-467e-4643-823d-d4b9186a4736 None admin] RBAC: Adding query filter params (name=member) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:00.887295 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662e9602-467e-4643-823d-d4b9186a4736 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:00.888710 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662e9602-467e-4643-823d-d4b9186a4736 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:00.890613 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:00.897040 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 91/177] 10.222.0.15 () {56 vars in 1101 bytes} [Sun Sep 11 10:33:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:00.897830 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662e9602-467e-4643-823d-d4b9186a4736 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:00.901722 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:00.940771 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 87/178] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:33:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:33:00.941810 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb89c4b7-d2bd-4913-b206-dd4aac0f71f6 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:00.942097 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb89c4b7-d2bd-4913-b206-dd4aac0f71f6 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:00.942370 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb89c4b7-d2bd-4913-b206-dd4aac0f71f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:00.960070 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-fb89c4b7-d2bd-4913-b206-dd4aac0f71f6 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:33:00.962343 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-fb89c4b7-d2bd-4913-b206-dd4aac0f71f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:33:01.015517 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662e9602-467e-4643-823d-d4b9186a4736 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:01.034867 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 88/179] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:33:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:33:01.036166 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ed164f-8eae-492a-b3c9-336345c3c150 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:01.036461 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ed164f-8eae-492a-b3c9-336345c3c150 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:01.036738 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ed164f-8eae-492a-b3c9-336345c3c150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:01.051019 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63b0d62-0c01-4cd7-b285-878cd27227f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:01.052718 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63b0d62-0c01-4cd7-b285-878cd27227f3 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:01.053204 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63b0d62-0c01-4cd7-b285-878cd27227f3 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:01.053715 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63b0d62-0c01-4cd7-b285-878cd27227f3 None admin] PATH_INFO: `/v3/projects/1b12f8d03f4f4f4c8b5f37b804de18c5` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:01.055064 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63b0d62-0c01-4cd7-b285-878cd27227f3 None admin] RBAC: Authorizing `identity:get_project(project_id=1b12f8d03f4f4f4c8b5f37b804de18c5)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:01.057682 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63b0d62-0c01-4cd7-b285-878cd27227f3 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:01.064619 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 92/180] 10.222.0.15 () {56 vars in 1186 bytes} [Sun Sep 11 10:33:00 2022] GET /identity/v3/projects/1b12f8d03f4f4f4c8b5f37b804de18c5 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:01.065231 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917d0b62-2ffb-4429-b92b-a2d51cc5461f None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:01.065930 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-b5ed164f-8eae-492a-b3c9-336345c3c150 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:33:01.067113 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-b5ed164f-8eae-492a-b3c9-336345c3c150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:33:01.067771 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917d0b62-2ffb-4429-b92b-a2d51cc5461f None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:01.068047 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917d0b62-2ffb-4429-b92b-a2d51cc5461f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:01.091539 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-917d0b62-2ffb-4429-b92b-a2d51cc5461f None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:33:01.093104 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-917d0b62-2ffb-4429-b92b-a2d51cc5461f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:33:01.152111 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 89/181] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:33:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:33:01.153324 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ed164f-8eae-492a-b3c9-336345c3c150 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:01.168920 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:01.179042 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 93/182] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:33:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:33:01.180294 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917d0b62-2ffb-4429-b92b-a2d51cc5461f None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:01.183543 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:01.260198 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ed164f-8eae-492a-b3c9-336345c3c150 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:01.293890 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917d0b62-2ffb-4429-b92b-a2d51cc5461f None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:01.332970 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021aa6dc-5caa-4065-a6e8-3fc623742eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:01.333944 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021aa6dc-5caa-4065-a6e8-3fc623742eda None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:01.334423 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021aa6dc-5caa-4065-a6e8-3fc623742eda None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:01.334892 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021aa6dc-5caa-4065-a6e8-3fc623742eda None admin] PATH_INFO: `/v3/users/0175a3d1c7824b08b476a3f905aad38e` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:01.336331 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021aa6dc-5caa-4065-a6e8-3fc623742eda None admin] RBAC: Authorizing `identity:get_user(user_id=0175a3d1c7824b08b476a3f905aad38e)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:01.338491 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021aa6dc-5caa-4065-a6e8-3fc623742eda None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:01.340315 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 94/183] 10.222.0.15 () {56 vars in 1177 bytes} [Sun Sep 11 10:33:01 2022] GET /identity/v3/users/0175a3d1c7824b08b476a3f905aad38e => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:01.340974 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d37d1bf-9dee-4836-b6c8-097bf2ffd219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:01.341505 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021aa6dc-5caa-4065-a6e8-3fc623742eda None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:01.341624 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d37d1bf-9dee-4836-b6c8-097bf2ffd219 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:01.341860 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d37d1bf-9dee-4836-b6c8-097bf2ffd219 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:01.342131 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d37d1bf-9dee-4836-b6c8-097bf2ffd219 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:01.342971 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d37d1bf-9dee-4836-b6c8-097bf2ffd219 None admin] RBAC: Adding query filter params (role.id=f836487a66c74c05838133c9d0614bbf, scope.project.id=1b12f8d03f4f4f4c8b5f37b804de18c5, user.id=0175a3d1c7824b08b476a3f905aad38e) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:01.343273 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d37d1bf-9dee-4836-b6c8-097bf2ffd219 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:01.344730 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d37d1bf-9dee-4836-b6c8-097bf2ffd219 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:01.358017 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 90/184] 10.222.0.15 () {56 vars in 1374 bytes} [Sun Sep 11 10:33:01 2022] GET /identity/v3/role_assignments?user.id=0175a3d1c7824b08b476a3f905aad38e&scope.project.id=1b12f8d03f4f4f4c8b5f37b804de18c5&role.id=f836487a66c74c05838133c9d0614bbf => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:01.358828 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d37d1bf-9dee-4836-b6c8-097bf2ffd219 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:01.422582 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:01.459767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021aa6dc-5caa-4065-a6e8-3fc623742eda None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:01.498406 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d281bf4f-7394-4d84-a06e-0ae023506a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:01.499412 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d281bf4f-7394-4d84-a06e-0ae023506a88 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:01.499728 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d281bf4f-7394-4d84-a06e-0ae023506a88 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:01.500004 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d281bf4f-7394-4d84-a06e-0ae023506a88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:01.500739 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d281bf4f-7394-4d84-a06e-0ae023506a88 None admin] RBAC: Adding query filter params (role.id=2cb7d73f0ea8414296367573f6902340, scope.project.id=1b12f8d03f4f4f4c8b5f37b804de18c5, user.id=3570ee53f3c442848938999c912a848b) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:01.501048 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d281bf4f-7394-4d84-a06e-0ae023506a88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:01.502499 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d281bf4f-7394-4d84-a06e-0ae023506a88 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:01.509994 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 95/185] 10.222.0.15 () {56 vars in 1374 bytes} [Sun Sep 11 10:33:01 2022] GET /identity/v3/role_assignments?user.id=3570ee53f3c442848938999c912a848b&scope.project.id=1b12f8d03f4f4f4c8b5f37b804de18c5&role.id=2cb7d73f0ea8414296367573f6902340 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:01.510984 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d281bf4f-7394-4d84-a06e-0ae023506a88 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:01.557228 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:01.588629 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d37d1bf-9dee-4836-b6c8-097bf2ffd219 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:01.635708 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bead47e5-b486-4ecd-8ec0-79156c5d22c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:01.636378 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead47e5-b486-4ecd-8ec0-79156c5d22c5 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:01.636662 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead47e5-b486-4ecd-8ec0-79156c5d22c5 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:01.636953 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead47e5-b486-4ecd-8ec0-79156c5d22c5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:01.642984 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d281bf4f-7394-4d84-a06e-0ae023506a88 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:01.650362 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bead47e5-b486-4ecd-8ec0-79156c5d22c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:33:01.656105 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-bead47e5-b486-4ecd-8ec0-79156c5d22c5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 10:33:01.656105 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:01.656105 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:01.656105 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:01.656105 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:01.656105 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:01.656105 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:01.656105 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:01.656105 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:01.656105 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:01.656105 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:01.656105 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:01.656105 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:33:01.656105 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:33:01.656105 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:33:01.656105 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:33:01.657057 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:33:01.657057 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:33:01.657057 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:33:01.657057 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:33:01.657057 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:01.657057 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:01.657057 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:01.657057 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:01.657057 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:01.657057 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:01.657057 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:01.657057 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:01.657057 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:01.657057 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:01.657057 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:01.657057 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:01.658283 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:01.658283 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:01.658283 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:01.658283 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:01.658283 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:33:01.658283 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:33:01.658283 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:33:01.658283 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:33:01.658283 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:33:01.658283 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:33:01.658283 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 10:33:01.658283 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:33:01.658283 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 91/186] 10.222.0.15 () {56 vars in 1099 bytes} [Sun Sep 11 10:33:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:01.658283 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bead47e5-b486-4ecd-8ec0-79156c5d22c5 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:01.700758 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9280a7a6-ec3a-41e1-9f37-cc53fce80201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:01.701755 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9280a7a6-ec3a-41e1-9f37-cc53fce80201 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:01.702043 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9280a7a6-ec3a-41e1-9f37-cc53fce80201 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:01.702317 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9280a7a6-ec3a-41e1-9f37-cc53fce80201 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:01.705550 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:01.792083 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9280a7a6-ec3a-41e1-9f37-cc53fce80201 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:01.793629 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9280a7a6-ec3a-41e1-9f37-cc53fce80201 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:01.808713 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bead47e5-b486-4ecd-8ec0-79156c5d22c5 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:01.842681 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ef3137-7dac-4799-a8b7-bdb5f1582bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:01.843310 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef3137-7dac-4799-a8b7-bdb5f1582bc5 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:01.843623 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef3137-7dac-4799-a8b7-bdb5f1582bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:01.843897 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef3137-7dac-4799-a8b7-bdb5f1582bc5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:01.845003 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ef3137-7dac-4799-a8b7-bdb5f1582bc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:01.846459 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ef3137-7dac-4799-a8b7-bdb5f1582bc5 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:01.848059 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 92/187] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:33:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:01.852984 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ef3137-7dac-4799-a8b7-bdb5f1582bc5 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:01.861807 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:01.891436 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 96/188] 10.222.0.15 () {58 vars in 1282 bytes} [Sun Sep 11 10:33:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 10:33:01.896220 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9280a7a6-ec3a-41e1-9f37-cc53fce80201 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:01.925054 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:01.956156 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ef3137-7dac-4799-a8b7-bdb5f1582bc5 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:01.989505 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9280a7a6-ec3a-41e1-9f37-cc53fce80201 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:01.989694 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0434f52-ea7d-4793-b159-9489ce5d47e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:01.990171 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0434f52-ea7d-4793-b159-9489ce5d47e4 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:01.990446 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0434f52-ea7d-4793-b159-9489ce5d47e4 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:01.990715 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0434f52-ea7d-4793-b159-9489ce5d47e4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:02.002036 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0434f52-ea7d-4793-b159-9489ce5d47e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:33:02.007185 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-b0434f52-ea7d-4793-b159-9489ce5d47e4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 10:33:02.007185 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:02.007185 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:02.007185 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:02.007185 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:02.007185 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:02.007185 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:02.007185 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:02.007185 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:02.007185 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:02.007185 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:02.007185 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:02.007185 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:33:02.007185 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:33:02.007185 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:33:02.007185 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:33:02.008086 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:33:02.008086 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:33:02.008086 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:33:02.008086 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:33:02.008086 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:02.008086 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:02.008086 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:02.008086 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:02.008086 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:02.008086 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:02.008086 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:02.008086 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:02.008086 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:02.008086 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:02.008086 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:02.008086 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:02.009446 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:02.009446 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:02.009446 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:02.009446 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:02.009446 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:33:02.009446 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:33:02.009446 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:33:02.009446 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:33:02.009446 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:33:02.009446 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:33:02.009446 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 10:33:02.009446 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:33:02.009446 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 93/189] 10.222.0.15 () {56 vars in 1114 bytes} [Sun Sep 11 10:33:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:02.011558 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0434f52-ea7d-4793-b159-9489ce5d47e4 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:02.024673 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0bfabd-f70f-4842-8b39-ce97e6f33bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:02.025300 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0bfabd-f70f-4842-8b39-ce97e6f33bcb None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:02.025576 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0bfabd-f70f-4842-8b39-ce97e6f33bcb None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:02.025845 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0bfabd-f70f-4842-8b39-ce97e6f33bcb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:02.035574 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0bfabd-f70f-4842-8b39-ce97e6f33bcb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:33:02.040503 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-9f0bfabd-f70f-4842-8b39-ce97e6f33bcb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 10:33:02.040503 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:02.040503 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:02.040503 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:02.040503 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:02.040503 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:02.040503 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:02.040503 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:02.040503 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:02.040503 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:02.040503 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:02.040503 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:02.040503 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:33:02.040503 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:33:02.040503 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:33:02.040503 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:33:02.041386 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:33:02.041386 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:33:02.041386 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:33:02.041386 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:33:02.041386 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:02.041386 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:02.041386 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:02.041386 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:02.041386 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:02.041386 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:02.041386 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:02.041386 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:02.041386 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:02.041386 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:02.041386 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:02.041386 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:02.042167 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:02.042167 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:02.042167 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:02.042167 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:02.042167 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:33:02.042167 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:33:02.042167 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:33:02.042167 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:33:02.042167 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:33:02.042167 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:33:02.042167 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 10:33:02.042167 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:33:02.042167 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 97/190] 10.222.0.15 () {56 vars in 1096 bytes} [Sun Sep 11 10:33:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:02.042811 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0bfabd-f70f-4842-8b39-ce97e6f33bcb None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:02.080473 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:02.115846 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0434f52-ea7d-4793-b159-9489ce5d47e4 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:02.134890 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:02.141919 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0bfabd-f70f-4842-8b39-ce97e6f33bcb None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:02.149717 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2908aa71-9c0e-452a-83ae-004f3a9d850b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:02.150492 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2908aa71-9c0e-452a-83ae-004f3a9d850b None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:02.150770 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2908aa71-9c0e-452a-83ae-004f3a9d850b None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:02.151039 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2908aa71-9c0e-452a-83ae-004f3a9d850b None admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:02.151726 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2908aa71-9c0e-452a-83ae-004f3a9d850b None admin] RBAC: Adding query filter params (name=reader) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:02.152026 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2908aa71-9c0e-452a-83ae-004f3a9d850b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:02.153400 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2908aa71-9c0e-452a-83ae-004f3a9d850b None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:02.161239 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 94/191] 10.222.0.15 () {56 vars in 1101 bytes} [Sun Sep 11 10:33:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:02.162063 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2908aa71-9c0e-452a-83ae-004f3a9d850b None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:02.179281 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1780fd60-8e0b-4f70-aab4-a4b9b13d0d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:02.179931 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1780fd60-8e0b-4f70-aab4-a4b9b13d0d07 None admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:02.180206 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1780fd60-8e0b-4f70-aab4-a4b9b13d0d07 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:02.180510 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1780fd60-8e0b-4f70-aab4-a4b9b13d0d07 None admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:02.181539 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1780fd60-8e0b-4f70-aab4-a4b9b13d0d07 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:02.182973 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1780fd60-8e0b-4f70-aab4-a4b9b13d0d07 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:02.204244 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 98/192] 10.222.0.15 () {60 vars in 1132 bytes} [Sun Sep 11 10:33:02 2022] POST /identity/v3/users => generated 297 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:33:02.205054 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1780fd60-8e0b-4f70-aab4-a4b9b13d0d07 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:02.291880 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2908aa71-9c0e-452a-83ae-004f3a9d850b None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:02.324462 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1780fd60-8e0b-4f70-aab4-a4b9b13d0d07 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:02.325649 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:02.334421 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38459c0d-44c7-4592-882b-95a628c7175e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:02.335078 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38459c0d-44c7-4592-882b-95a628c7175e None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:02.335363 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38459c0d-44c7-4592-882b-95a628c7175e None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:02.335666 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38459c0d-44c7-4592-882b-95a628c7175e None admin] PATH_INFO: `/v3/projects/b21540a9dc144b76bf6e2e819b54cccf` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:02.337425 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38459c0d-44c7-4592-882b-95a628c7175e None admin] RBAC: Authorizing `identity:get_project(project_id=b21540a9dc144b76bf6e2e819b54cccf)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:02.339252 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38459c0d-44c7-4592-882b-95a628c7175e None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:02.345962 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 95/193] 10.222.0.15 () {56 vars in 1186 bytes} [Sun Sep 11 10:33:02 2022] GET /identity/v3/projects/b21540a9dc144b76bf6e2e819b54cccf => generated 312 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:02.345962 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38459c0d-44c7-4592-882b-95a628c7175e None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:02.370772 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cd5598-453f-4cc3-a355-54297bf2c33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:02.371402 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cd5598-453f-4cc3-a355-54297bf2c33d None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:02.371730 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cd5598-453f-4cc3-a355-54297bf2c33d None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:02.372003 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cd5598-453f-4cc3-a355-54297bf2c33d None admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:02.372688 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cd5598-453f-4cc3-a355-54297bf2c33d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:02.372979 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cd5598-453f-4cc3-a355-54297bf2c33d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:02.374442 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cd5598-453f-4cc3-a355-54297bf2c33d None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:02.383067 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 99/194] 10.222.0.15 () {56 vars in 1111 bytes} [Sun Sep 11 10:33:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:02.384470 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cd5598-453f-4cc3-a355-54297bf2c33d None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:02.401045 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:02.467084 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38459c0d-44c7-4592-882b-95a628c7175e None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:02.510971 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cd5598-453f-4cc3-a355-54297bf2c33d None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:02.512856 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5c42f8-8cfc-41dc-83a7-2c25e8dc62e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:02.513483 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5c42f8-8cfc-41dc-83a7-2c25e8dc62e4 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:02.513750 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5c42f8-8cfc-41dc-83a7-2c25e8dc62e4 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:02.514027 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5c42f8-8cfc-41dc-83a7-2c25e8dc62e4 None admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:02.514694 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5c42f8-8cfc-41dc-83a7-2c25e8dc62e4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:02.514997 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5c42f8-8cfc-41dc-83a7-2c25e8dc62e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:02.516429 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5c42f8-8cfc-41dc-83a7-2c25e8dc62e4 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:02.531552 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 96/195] 10.222.0.15 () {56 vars in 1099 bytes} [Sun Sep 11 10:33:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:02.531552 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5c42f8-8cfc-41dc-83a7-2c25e8dc62e4 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:02.563499 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3718f4-2690-401a-a756-d77d9a0e6026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:02.564084 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3718f4-2690-401a-a756-d77d9a0e6026 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:02.564363 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3718f4-2690-401a-a756-d77d9a0e6026 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:02.564633 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3718f4-2690-401a-a756-d77d9a0e6026 None admin] PATH_INFO: `/v3/users/53e4b2016de34e7bb3adfa8fa78a1610` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:02.566364 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3718f4-2690-401a-a756-d77d9a0e6026 None admin] RBAC: Authorizing `identity:get_user(user_id=53e4b2016de34e7bb3adfa8fa78a1610)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:02.568444 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3718f4-2690-401a-a756-d77d9a0e6026 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:02.570721 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 100/196] 10.222.0.15 () {56 vars in 1177 bytes} [Sun Sep 11 10:33:02 2022] GET /identity/v3/users/53e4b2016de34e7bb3adfa8fa78a1610 => generated 289 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:02.572158 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3718f4-2690-401a-a756-d77d9a0e6026 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:02.613286 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:02.682667 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5c42f8-8cfc-41dc-83a7-2c25e8dc62e4 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:02.683197 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:02.703845 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3718f4-2690-401a-a756-d77d9a0e6026 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:02.730736 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb3719b-076c-4bae-b227-d31d8bf49677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:02.732246 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb3719b-076c-4bae-b227-d31d8bf49677 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:02.732693 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb3719b-076c-4bae-b227-d31d8bf49677 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:02.733353 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb3719b-076c-4bae-b227-d31d8bf49677 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:02.734313 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb3719b-076c-4bae-b227-d31d8bf49677 None admin] RBAC: Adding query filter params (role.id=f836487a66c74c05838133c9d0614bbf, scope.project.id=b21540a9dc144b76bf6e2e819b54cccf, user.id=0175a3d1c7824b08b476a3f905aad38e) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:02.734756 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb3719b-076c-4bae-b227-d31d8bf49677 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:02.736453 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb3719b-076c-4bae-b227-d31d8bf49677 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:02.740080 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17158e55-a560-4d88-88c9-8deea2ef4e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:02.740789 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17158e55-a560-4d88-88c9-8deea2ef4e80 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:02.741155 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17158e55-a560-4d88-88c9-8deea2ef4e80 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:02.741502 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17158e55-a560-4d88-88c9-8deea2ef4e80 None admin] PATH_INFO: `/v3/users/f5bed6f6065e483987356948f5be4b73` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:02.746376 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 97/197] 10.222.0.15 () {56 vars in 1374 bytes} [Sun Sep 11 10:33:02 2022] GET /identity/v3/role_assignments?user.id=0175a3d1c7824b08b476a3f905aad38e&scope.project.id=b21540a9dc144b76bf6e2e819b54cccf&role.id=f836487a66c74c05838133c9d0614bbf => generated 608 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:02.747467 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb3719b-076c-4bae-b227-d31d8bf49677 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:02.757525 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17158e55-a560-4d88-88c9-8deea2ef4e80 None admin] RBAC: Authorizing `identity:get_user(user_id=f5bed6f6065e483987356948f5be4b73)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:02.759774 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17158e55-a560-4d88-88c9-8deea2ef4e80 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:02.761316 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 101/198] 10.222.0.15 () {56 vars in 1177 bytes} [Sun Sep 11 10:33:02 2022] GET /identity/v3/users/f5bed6f6065e483987356948f5be4b73 => generated 283 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:02.762103 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17158e55-a560-4d88-88c9-8deea2ef4e80 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:02.877904 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb3719b-076c-4bae-b227-d31d8bf49677 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:02.905909 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:02.936547 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1281c7-46a4-4f8d-bc4c-61278d178da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:02.937214 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1281c7-46a4-4f8d-bc4c-61278d178da7 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:02.937495 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1281c7-46a4-4f8d-bc4c-61278d178da7 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:02.937768 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1281c7-46a4-4f8d-bc4c-61278d178da7 None admin] PATH_INFO: `/v3/users/f5bed6f6065e483987356948f5be4b73` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:02.939242 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1281c7-46a4-4f8d-bc4c-61278d178da7 None admin] RBAC: Authorizing `identity:get_user(user_id=f5bed6f6065e483987356948f5be4b73)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:02.940735 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1281c7-46a4-4f8d-bc4c-61278d178da7 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:02.942384 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 98/199] 10.222.0.15 () {56 vars in 1177 bytes} [Sun Sep 11 10:33:02 2022] GET /identity/v3/users/f5bed6f6065e483987356948f5be4b73 => generated 283 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:02.943171 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1281c7-46a4-4f8d-bc4c-61278d178da7 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:02.995627 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17158e55-a560-4d88-88c9-8deea2ef4e80 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:03.044273 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd961663-90fb-49aa-85e1-271b557b87c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:03.044895 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd961663-90fb-49aa-85e1-271b557b87c6 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:03.045176 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd961663-90fb-49aa-85e1-271b557b87c6 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:03.045450 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd961663-90fb-49aa-85e1-271b557b87c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:03.061264 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:03.108709 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1281c7-46a4-4f8d-bc4c-61278d178da7 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:03.136135 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd961663-90fb-49aa-85e1-271b557b87c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:03.137171 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd961663-90fb-49aa-85e1-271b557b87c6 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:03.153210 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e1765f-e277-45db-a545-12f8ad6fd07a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:03.154200 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e1765f-e277-45db-a545-12f8ad6fd07a None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:03.154829 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e1765f-e277-45db-a545-12f8ad6fd07a None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:03.155614 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e1765f-e277-45db-a545-12f8ad6fd07a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:03.232827 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:03.247356 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e1765f-e277-45db-a545-12f8ad6fd07a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:03.249223 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e1765f-e277-45db-a545-12f8ad6fd07a None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:03.251642 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 102/200] 10.222.0.15 () {58 vars in 1282 bytes} [Sun Sep 11 10:33:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 10:33:03.253160 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd961663-90fb-49aa-85e1-271b557b87c6 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:03.302494 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:03.389465 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd961663-90fb-49aa-85e1-271b557b87c6 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:03.390228 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 99/201] 10.222.0.15 () {58 vars in 1282 bytes} [Sun Sep 11 10:33:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 10:33:03.391277 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e1765f-e277-45db-a545-12f8ad6fd07a None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:03.431350 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c71613a-9581-440e-ab6b-e72f110dcac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:03.432122 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c71613a-9581-440e-ab6b-e72f110dcac3 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:03.432405 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c71613a-9581-440e-ab6b-e72f110dcac3 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:03.432681 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c71613a-9581-440e-ab6b-e72f110dcac3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:03.505565 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:03.507783 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c71613a-9581-440e-ab6b-e72f110dcac3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:03.509573 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c71613a-9581-440e-ab6b-e72f110dcac3 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:03.517818 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e1765f-e277-45db-a545-12f8ad6fd07a None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:03.551733 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:03.562900 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243db3c2-c9ac-41dd-80a7-449ef2aba373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:03.563966 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243db3c2-c9ac-41dd-80a7-449ef2aba373 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:03.564695 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243db3c2-c9ac-41dd-80a7-449ef2aba373 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:03.565145 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243db3c2-c9ac-41dd-80a7-449ef2aba373 None admin] PATH_INFO: `/v3/projects/1b12f8d03f4f4f4c8b5f37b804de18c5` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:03.566455 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243db3c2-c9ac-41dd-80a7-449ef2aba373 None admin] RBAC: Authorizing `identity:get_project(project_id=1b12f8d03f4f4f4c8b5f37b804de18c5)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:03.569351 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243db3c2-c9ac-41dd-80a7-449ef2aba373 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:03.571299 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 100/202] 10.222.0.15 () {56 vars in 1186 bytes} [Sun Sep 11 10:33:03 2022] GET /identity/v3/projects/1b12f8d03f4f4f4c8b5f37b804de18c5 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:03.572071 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243db3c2-c9ac-41dd-80a7-449ef2aba373 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:03.624927 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 103/203] 10.222.0.15 () {58 vars in 1282 bytes} [Sun Sep 11 10:33:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 10:33:03.630380 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c71613a-9581-440e-ab6b-e72f110dcac3 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:03.694216 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243db3c2-c9ac-41dd-80a7-449ef2aba373 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:03.745033 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c71613a-9581-440e-ab6b-e72f110dcac3 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:03.745753 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cb80cc-944c-4796-b630-d31612315dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:03.746821 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb80cc-944c-4796-b630-d31612315dd8 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:03.748174 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb80cc-944c-4796-b630-d31612315dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:03.748601 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb80cc-944c-4796-b630-d31612315dd8 None admin] PATH_INFO: `/v3/projects/6d140898a8684238bddc9872a32eb94c` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:03.749980 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cb80cc-944c-4796-b630-d31612315dd8 None admin] RBAC: Authorizing `identity:get_project(project_id=6d140898a8684238bddc9872a32eb94c)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:03.751662 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cb80cc-944c-4796-b630-d31612315dd8 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:03.753992 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 101/204] 10.222.0.15 () {56 vars in 1186 bytes} [Sun Sep 11 10:33:03 2022] GET /identity/v3/projects/6d140898a8684238bddc9872a32eb94c => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:03.755368 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cb80cc-944c-4796-b630-d31612315dd8 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:03.781571 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a45e2f-5a10-4689-a8af-e91f9b51b75b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:03.782250 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a45e2f-5a10-4689-a8af-e91f9b51b75b None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:03.782445 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a45e2f-5a10-4689-a8af-e91f9b51b75b None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:03.782714 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a45e2f-5a10-4689-a8af-e91f9b51b75b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:03.783459 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a45e2f-5a10-4689-a8af-e91f9b51b75b None admin] RBAC: Adding query filter params (role.id=d577f4ba2f9c412e9530e50d7f3d6bd7, scope.project.id=1b12f8d03f4f4f4c8b5f37b804de18c5, user.id=53e4b2016de34e7bb3adfa8fa78a1610) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:03.783774 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a45e2f-5a10-4689-a8af-e91f9b51b75b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:03.785212 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a45e2f-5a10-4689-a8af-e91f9b51b75b None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:03.793351 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:03.798248 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 104/205] 10.222.0.15 () {56 vars in 1374 bytes} [Sun Sep 11 10:33:03 2022] GET /identity/v3/role_assignments?user.id=53e4b2016de34e7bb3adfa8fa78a1610&scope.project.id=1b12f8d03f4f4f4c8b5f37b804de18c5&role.id=d577f4ba2f9c412e9530e50d7f3d6bd7 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:03.799341 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a45e2f-5a10-4689-a8af-e91f9b51b75b None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:03.808044 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:03.945799 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cb80cc-944c-4796-b630-d31612315dd8 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:03.992694 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ccbd27-a3a1-4c1c-9e28-d871d883e6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:03.994408 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ccbd27-a3a1-4c1c-9e28-d871d883e6d6 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:03.994887 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ccbd27-a3a1-4c1c-9e28-d871d883e6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:03.995376 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ccbd27-a3a1-4c1c-9e28-d871d883e6d6 None admin] PATH_INFO: `/v3/projects/6d140898a8684238bddc9872a32eb94c` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:03.997058 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ccbd27-a3a1-4c1c-9e28-d871d883e6d6 None admin] RBAC: Authorizing `identity:get_project(project_id=6d140898a8684238bddc9872a32eb94c)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:03.998817 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ccbd27-a3a1-4c1c-9e28-d871d883e6d6 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:04.001266 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 102/206] 10.222.0.15 () {56 vars in 1186 bytes} [Sun Sep 11 10:33:03 2022] GET /identity/v3/projects/6d140898a8684238bddc9872a32eb94c => generated 302 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:04.007275 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a45e2f-5a10-4689-a8af-e91f9b51b75b None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:04.009222 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ccbd27-a3a1-4c1c-9e28-d871d883e6d6 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:04.058460 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01580915-198b-455c-94f8-f57774e5ef6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:04.058460 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01580915-198b-455c-94f8-f57774e5ef6f None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:04.058460 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01580915-198b-455c-94f8-f57774e5ef6f None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:04.058792 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01580915-198b-455c-94f8-f57774e5ef6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:04.059461 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01580915-198b-455c-94f8-f57774e5ef6f None admin] RBAC: Adding query filter params (role.id=cdb3aa1bd85b4694964629ff7cf8039b, scope.project.id=6d140898a8684238bddc9872a32eb94c, user.id=f5bed6f6065e483987356948f5be4b73) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:04.059772 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01580915-198b-455c-94f8-f57774e5ef6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:04.062177 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01580915-198b-455c-94f8-f57774e5ef6f None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:04.068530 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 105/207] 10.222.0.15 () {56 vars in 1374 bytes} [Sun Sep 11 10:33:03 2022] GET /identity/v3/role_assignments?user.id=f5bed6f6065e483987356948f5be4b73&scope.project.id=6d140898a8684238bddc9872a32eb94c&role.id=cdb3aa1bd85b4694964629ff7cf8039b => generated 259 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:04.131532 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:04.140308 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ccbd27-a3a1-4c1c-9e28-d871d883e6d6 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:04.182520 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aab2624-3005-4e01-bc85-71dcd68e9e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:04.183626 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aab2624-3005-4e01-bc85-71dcd68e9e8a None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:04.184114 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aab2624-3005-4e01-bc85-71dcd68e9e8a None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:04.184580 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aab2624-3005-4e01-bc85-71dcd68e9e8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:04.185551 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aab2624-3005-4e01-bc85-71dcd68e9e8a None admin] RBAC: Adding query filter params (role.id=2cb7d73f0ea8414296367573f6902340, scope.project.id=6d140898a8684238bddc9872a32eb94c, user.id=f5bed6f6065e483987356948f5be4b73) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:04.186094 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aab2624-3005-4e01-bc85-71dcd68e9e8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:04.187841 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aab2624-3005-4e01-bc85-71dcd68e9e8a None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:04.194912 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 103/208] 10.222.0.15 () {56 vars in 1374 bytes} [Sun Sep 11 10:33:04 2022] GET /identity/v3/role_assignments?user.id=f5bed6f6065e483987356948f5be4b73&scope.project.id=6d140898a8684238bddc9872a32eb94c&role.id=2cb7d73f0ea8414296367573f6902340 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:04.709908 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63805644-1435-4f5f-a08d-ecc15a1a6678 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:04.710397 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63805644-1435-4f5f-a08d-ecc15a1a6678 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:04.710397 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63805644-1435-4f5f-a08d-ecc15a1a6678 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:04.712217 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 106/209] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:33:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:33:04.717786 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea9d41b-6205-4daf-a118-940399b17688 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:04.717931 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6962af97-7430-4755-9785-fb7f7d4613c1 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:04.718315 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6962af97-7430-4755-9785-fb7f7d4613c1 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:04.718398 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6962af97-7430-4755-9785-fb7f7d4613c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:04.718918 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea9d41b-6205-4daf-a118-940399b17688 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:04.719402 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea9d41b-6205-4daf-a118-940399b17688 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:04.720901 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 107/210] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:33:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 10:33:04.725866 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda320e-271f-407d-83f3-2b6baace68b5 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:04.726151 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda320e-271f-407d-83f3-2b6baace68b5 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:04.726684 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda320e-271f-407d-83f3-2b6baace68b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:04.739653 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-6962af97-7430-4755-9785-fb7f7d4613c1 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:33:04.741189 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-6962af97-7430-4755-9785-fb7f7d4613c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:33:04.745552 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-edda320e-271f-407d-83f3-2b6baace68b5 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:33:04.746721 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-edda320e-271f-407d-83f3-2b6baace68b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:33:04.814346 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:04.815552 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 104/211] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:33:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:33:04.826511 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 108/212] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:33:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 10:33:05.084959 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6962af97-7430-4755-9785-fb7f7d4613c1 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:05.100352 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda320e-271f-407d-83f3-2b6baace68b5 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:05.195765 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:05.206557 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda320e-271f-407d-83f3-2b6baace68b5 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:05.207262 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6962af97-7430-4755-9785-fb7f7d4613c1 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:05.245424 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77970d11-f198-4e3f-9c8f-763d1775bb4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:05.246054 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77970d11-f198-4e3f-9c8f-763d1775bb4c None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:05.246329 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77970d11-f198-4e3f-9c8f-763d1775bb4c None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:05.246599 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77970d11-f198-4e3f-9c8f-763d1775bb4c None admin] PATH_INFO: `/v3/roles/member` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:05.250671 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28dafca-48b9-4f39-9037-17a6bfd7b669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:05.251507 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28dafca-48b9-4f39-9037-17a6bfd7b669 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:05.251915 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28dafca-48b9-4f39-9037-17a6bfd7b669 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:05.252202 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28dafca-48b9-4f39-9037-17a6bfd7b669 None admin] PATH_INFO: `/v3/domains/default` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:05.253595 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28dafca-48b9-4f39-9037-17a6bfd7b669 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:05.255161 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28dafca-48b9-4f39-9037-17a6bfd7b669 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:05.257484 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77970d11-f198-4e3f-9c8f-763d1775bb4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:33:05.261966 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 105/213] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:33:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:05.262766 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28dafca-48b9-4f39-9037-17a6bfd7b669 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:05.263996 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-77970d11-f198-4e3f-9c8f-763d1775bb4c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 10:33:05.263996 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:05.263996 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:05.263996 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:05.263996 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:05.263996 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:05.263996 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:05.263996 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:05.263996 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:05.263996 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:05.263996 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:05.263996 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:05.263996 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:33:05.263996 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:33:05.263996 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:33:05.263996 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:33:05.265038 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:33:05.265038 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:33:05.265038 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:33:05.265038 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:33:05.265038 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:05.265038 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:05.265038 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:05.265038 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:05.265038 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:05.265038 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:05.265038 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:05.265038 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:05.265038 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:05.265038 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:05.265038 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:05.265038 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:05.266043 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:05.266043 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:05.266043 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:05.266043 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:05.266043 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:33:05.266043 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:33:05.266043 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:33:05.266043 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:33:05.266043 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:33:05.266043 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:33:05.266043 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 10:33:05.266043 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:33:05.271445 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 109/214] 10.222.0.15 () {56 vars in 1099 bytes} [Sun Sep 11 10:33:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:05.273251 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77970d11-f198-4e3f-9c8f-763d1775bb4c None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:05.343543 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:05.384801 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28dafca-48b9-4f39-9037-17a6bfd7b669 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:05.391961 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77970d11-f198-4e3f-9c8f-763d1775bb4c None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:05.425998 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b01d5ab-da04-4466-9d8b-a41470c572d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:05.426730 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b01d5ab-da04-4466-9d8b-a41470c572d7 None admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:05.427089 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b01d5ab-da04-4466-9d8b-a41470c572d7 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:05.427373 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b01d5ab-da04-4466-9d8b-a41470c572d7 None admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:05.428256 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b01d5ab-da04-4466-9d8b-a41470c572d7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:05.429675 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b2d029-ab04-4989-94d7-49b3777d977a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:05.429914 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b01d5ab-da04-4466-9d8b-a41470c572d7 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:05.430650 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b2d029-ab04-4989-94d7-49b3777d977a None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:05.431253 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b2d029-ab04-4989-94d7-49b3777d977a None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:05.431651 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b2d029-ab04-4989-94d7-49b3777d977a None admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:05.432416 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b2d029-ab04-4989-94d7-49b3777d977a None admin] RBAC: Adding query filter params (name=member) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:05.432816 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b2d029-ab04-4989-94d7-49b3777d977a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:05.434277 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b2d029-ab04-4989-94d7-49b3777d977a None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:05.444003 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 110/215] 10.222.0.15 () {56 vars in 1101 bytes} [Sun Sep 11 10:33:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:05.451690 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b2d029-ab04-4989-94d7-49b3777d977a None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:05.456886 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 106/216] 10.222.0.15 () {60 vars in 1132 bytes} [Sun Sep 11 10:33:05 2022] POST /identity/v3/users => generated 297 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:33:05.569726 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b2d029-ab04-4989-94d7-49b3777d977a None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:05.588853 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:05.608825 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cd13f6-efb0-4448-a927-8926386d8adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:05.609389 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cd13f6-efb0-4448-a927-8926386d8adb None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:05.609673 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cd13f6-efb0-4448-a927-8926386d8adb None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:05.609943 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cd13f6-efb0-4448-a927-8926386d8adb None admin] PATH_INFO: `/v3/users/5e1ca49806f84e66a9cc647892bcea5c` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:05.626330 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cd13f6-efb0-4448-a927-8926386d8adb None admin] RBAC: Authorizing `identity:get_user(user_id=5e1ca49806f84e66a9cc647892bcea5c)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:05.627848 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cd13f6-efb0-4448-a927-8926386d8adb None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:05.634487 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 111/217] 10.222.0.15 () {56 vars in 1177 bytes} [Sun Sep 11 10:33:05 2022] GET /identity/v3/users/5e1ca49806f84e66a9cc647892bcea5c => generated 297 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:05.635975 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b01d5ab-da04-4466-9d8b-a41470c572d7 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:05.660094 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:05.752692 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b01d5ab-da04-4466-9d8b-a41470c572d7 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:05.788646 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dfc697-fb8f-418a-bbca-12870239ea90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:05.789262 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dfc697-fb8f-418a-bbca-12870239ea90 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:05.789539 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dfc697-fb8f-418a-bbca-12870239ea90 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:05.789949 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dfc697-fb8f-418a-bbca-12870239ea90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:05.863100 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dfc697-fb8f-418a-bbca-12870239ea90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:05.864532 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dfc697-fb8f-418a-bbca-12870239ea90 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:05.880353 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:05.988807 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 107/218] 10.222.0.15 () {58 vars in 1282 bytes} [Sun Sep 11 10:33:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 10:33:05.990750 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cd13f6-efb0-4448-a927-8926386d8adb None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:06.107265 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cd13f6-efb0-4448-a927-8926386d8adb None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:06.158697 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68435245-78b8-4036-824a-e3429919d080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:06.159262 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68435245-78b8-4036-824a-e3429919d080 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:06.159569 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68435245-78b8-4036-824a-e3429919d080 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:06.159839 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68435245-78b8-4036-824a-e3429919d080 None admin] PATH_INFO: `/v3/projects/6d140898a8684238bddc9872a32eb94c` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:06.160982 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68435245-78b8-4036-824a-e3429919d080 None admin] RBAC: Authorizing `identity:get_project(project_id=6d140898a8684238bddc9872a32eb94c)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:06.162410 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68435245-78b8-4036-824a-e3429919d080 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:06.164131 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 112/219] 10.222.0.15 () {56 vars in 1186 bytes} [Sun Sep 11 10:33:05 2022] GET /identity/v3/projects/6d140898a8684238bddc9872a32eb94c => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:06.170948 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dfc697-fb8f-418a-bbca-12870239ea90 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:06.306732 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dfc697-fb8f-418a-bbca-12870239ea90 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:06.322792 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:06.357901 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8842108b-6cba-4bca-a28e-b52d71264255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:06.358519 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8842108b-6cba-4bca-a28e-b52d71264255 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:06.358797 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8842108b-6cba-4bca-a28e-b52d71264255 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:06.359066 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8842108b-6cba-4bca-a28e-b52d71264255 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:06.359814 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8842108b-6cba-4bca-a28e-b52d71264255 None admin] RBAC: Adding query filter params (role.id=f836487a66c74c05838133c9d0614bbf, scope.project.id=6d140898a8684238bddc9872a32eb94c, user.id=5e1ca49806f84e66a9cc647892bcea5c) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:06.360116 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8842108b-6cba-4bca-a28e-b52d71264255 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:06.361551 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8842108b-6cba-4bca-a28e-b52d71264255 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:06.373398 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 108/220] 10.222.0.15 () {56 vars in 1374 bytes} [Sun Sep 11 10:33:06 2022] GET /identity/v3/role_assignments?user.id=5e1ca49806f84e66a9cc647892bcea5c&scope.project.id=6d140898a8684238bddc9872a32eb94c&role.id=f836487a66c74c05838133c9d0614bbf => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:06.453155 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae54807b-20d6-4c06-9cbc-5f9b7bc18110 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:06.453343 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae54807b-20d6-4c06-9cbc-5f9b7bc18110 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:06.453607 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae54807b-20d6-4c06-9cbc-5f9b7bc18110 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:06.455033 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 113/221] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:33:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:33:06.460703 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5fb0d9-0abd-4bfc-8cc1-4bea356e70e3 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:06.460988 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5fb0d9-0abd-4bfc-8cc1-4bea356e70e3 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:06.461248 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5fb0d9-0abd-4bfc-8cc1-4bea356e70e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:06.489372 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-1f5fb0d9-0abd-4bfc-8cc1-4bea356e70e3 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:33:06.490628 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-1f5fb0d9-0abd-4bfc-8cc1-4bea356e70e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:33:06.536963 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b738c-a84e-466e-903a-a3811e465a56 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:06.537285 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b738c-a84e-466e-903a-a3811e465a56 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:06.537517 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b738c-a84e-466e-903a-a3811e465a56 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:06.543451 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 114/222] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:33:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:33:06.562762 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ade146-e8c5-4498-b518-dd8f56495339 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:06.563062 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ade146-e8c5-4498-b518-dd8f56495339 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:06.563314 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ade146-e8c5-4498-b518-dd8f56495339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:06.591028 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-43ade146-e8c5-4498-b518-dd8f56495339 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:33:06.592284 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-43ade146-e8c5-4498-b518-dd8f56495339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:33:06.614405 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 109/223] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:33:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:33:06.657735 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:06.687718 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 115/224] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:33:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 10:33:06.961977 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5fb0d9-0abd-4bfc-8cc1-4bea356e70e3 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:07.007193 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ade146-e8c5-4498-b518-dd8f56495339 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:07.106567 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5fb0d9-0abd-4bfc-8cc1-4bea356e70e3 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:07.123946 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:07.169634 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ade146-e8c5-4498-b518-dd8f56495339 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:07.176099 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a48b4-b534-464d-9ff7-6c738f440b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:07.177046 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a48b4-b534-464d-9ff7-6c738f440b9c None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:07.177515 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a48b4-b534-464d-9ff7-6c738f440b9c None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:07.177972 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a48b4-b534-464d-9ff7-6c738f440b9c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:07.201690 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745a48b4-b534-464d-9ff7-6c738f440b9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:33:07.211090 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-745a48b4-b534-464d-9ff7-6c738f440b9c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 10:33:07.211090 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:07.211090 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:07.211090 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:07.211090 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:07.211090 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:07.211090 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:07.211090 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:07.211090 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:07.211090 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:07.211090 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:07.211090 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:07.211090 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:33:07.211090 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:33:07.211090 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:33:07.211090 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:33:07.212942 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:33:07.212942 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:33:07.212942 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:33:07.212942 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:33:07.212942 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:07.212942 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:07.212942 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:07.212942 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:07.212942 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:07.212942 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:07.212942 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:07.212942 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:07.212942 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:07.212942 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:07.212942 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:07.212942 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:07.213763 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:07.213763 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:07.213763 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:07.213763 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:07.213763 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:33:07.213763 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:33:07.213763 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:33:07.213763 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:33:07.213763 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:33:07.213763 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:33:07.213763 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 10:33:07.213763 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:33:07.215096 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 110/225] 10.222.0.15 () {56 vars in 1096 bytes} [Sun Sep 11 10:33:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:07.221573 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a48b4-b534-464d-9ff7-6c738f440b9c None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:07.222542 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09385434-9463-4a02-aa98-07a9ea517e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:07.223187 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09385434-9463-4a02-aa98-07a9ea517e4f None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:07.223494 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09385434-9463-4a02-aa98-07a9ea517e4f None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:07.223773 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09385434-9463-4a02-aa98-07a9ea517e4f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:07.247253 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09385434-9463-4a02-aa98-07a9ea517e4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:33:07.252216 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-09385434-9463-4a02-aa98-07a9ea517e4f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 10:33:07.252216 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:07.252216 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:07.252216 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:07.252216 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:07.252216 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:07.252216 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:07.252216 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:07.252216 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:07.252216 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:07.252216 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:07.252216 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:07.252216 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:33:07.252216 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:33:07.252216 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:33:07.252216 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:33:07.253091 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:33:07.253091 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:33:07.253091 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:33:07.253091 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:33:07.253091 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:07.253091 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:07.253091 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:07.253091 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:07.253091 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:07.253091 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:07.253091 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:07.253091 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:07.253091 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:07.253091 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:07.253091 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:07.253091 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:07.254573 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:07.254573 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:07.254573 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:07.254573 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:07.254573 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:33:07.254573 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:33:07.254573 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:33:07.254573 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:33:07.254573 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:33:07.254573 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:33:07.254573 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 10:33:07.254573 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:33:07.254573 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 116/226] 10.222.0.15 () {56 vars in 1114 bytes} [Sun Sep 11 10:33:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:07.260129 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09385434-9463-4a02-aa98-07a9ea517e4f None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:07.327774 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:07.422926 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a48b4-b534-464d-9ff7-6c738f440b9c None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:07.457380 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09385434-9463-4a02-aa98-07a9ea517e4f None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:07.471489 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72dbc9b-9988-4201-9459-5e6a3d07b7b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:07.472032 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72dbc9b-9988-4201-9459-5e6a3d07b7b2 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:07.472372 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72dbc9b-9988-4201-9459-5e6a3d07b7b2 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:07.472649 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72dbc9b-9988-4201-9459-5e6a3d07b7b2 None admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:07.473452 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72dbc9b-9988-4201-9459-5e6a3d07b7b2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:07.473762 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72dbc9b-9988-4201-9459-5e6a3d07b7b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:07.477625 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72dbc9b-9988-4201-9459-5e6a3d07b7b2 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:07.496394 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 111/227] 10.222.0.15 () {56 vars in 1099 bytes} [Sun Sep 11 10:33:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:07.503883 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72dbc9b-9988-4201-9459-5e6a3d07b7b2 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:07.530894 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2259f7a-7397-4e59-bde8-ce4fef000422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:07.531572 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2259f7a-7397-4e59-bde8-ce4fef000422 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:07.532550 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2259f7a-7397-4e59-bde8-ce4fef000422 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:07.532835 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2259f7a-7397-4e59-bde8-ce4fef000422 None admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:07.533503 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2259f7a-7397-4e59-bde8-ce4fef000422 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:07.533811 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2259f7a-7397-4e59-bde8-ce4fef000422 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:07.536926 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2259f7a-7397-4e59-bde8-ce4fef000422 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:07.552639 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 117/228] 10.222.0.15 () {56 vars in 1111 bytes} [Sun Sep 11 10:33:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:07.558705 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2259f7a-7397-4e59-bde8-ce4fef000422 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:07.583795 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:07.689523 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72dbc9b-9988-4201-9459-5e6a3d07b7b2 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:07.747078 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:07.751543 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32247dea-3896-4d3e-959f-36671368c625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:07.753124 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32247dea-3896-4d3e-959f-36671368c625 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:07.754012 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32247dea-3896-4d3e-959f-36671368c625 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:07.754477 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32247dea-3896-4d3e-959f-36671368c625 None admin] PATH_INFO: `/v3/users/f5bed6f6065e483987356948f5be4b73` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:07.757688 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2259f7a-7397-4e59-bde8-ce4fef000422 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:07.759106 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32247dea-3896-4d3e-959f-36671368c625 None admin] RBAC: Authorizing `identity:get_user(user_id=f5bed6f6065e483987356948f5be4b73)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:07.760981 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32247dea-3896-4d3e-959f-36671368c625 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:07.770165 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 112/229] 10.222.0.15 () {56 vars in 1177 bytes} [Sun Sep 11 10:33:07 2022] GET /identity/v3/users/f5bed6f6065e483987356948f5be4b73 => generated 283 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:07.775766 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32247dea-3896-4d3e-959f-36671368c625 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:07.845485 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c19fffb-7ba3-4c70-b9b1-c6b87a04077a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:07.846129 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c19fffb-7ba3-4c70-b9b1-c6b87a04077a None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:07.846407 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c19fffb-7ba3-4c70-b9b1-c6b87a04077a None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:07.846689 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c19fffb-7ba3-4c70-b9b1-c6b87a04077a None admin] PATH_INFO: `/v3/users/f5bed6f6065e483987356948f5be4b73` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:07.852363 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c19fffb-7ba3-4c70-b9b1-c6b87a04077a None admin] RBAC: Authorizing `identity:get_user(user_id=f5bed6f6065e483987356948f5be4b73)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:07.853832 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c19fffb-7ba3-4c70-b9b1-c6b87a04077a None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:07.855766 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 118/230] 10.222.0.15 () {56 vars in 1177 bytes} [Sun Sep 11 10:33:07 2022] GET /identity/v3/users/f5bed6f6065e483987356948f5be4b73 => generated 283 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:07.864361 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c19fffb-7ba3-4c70-b9b1-c6b87a04077a None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:07.895811 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:07.899995 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32247dea-3896-4d3e-959f-36671368c625 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:07.937774 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bedecd0-ba00-4081-90a2-4b491c9882c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:07.939280 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bedecd0-ba00-4081-90a2-4b491c9882c4 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:07.939877 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bedecd0-ba00-4081-90a2-4b491c9882c4 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:07.940354 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bedecd0-ba00-4081-90a2-4b491c9882c4 None admin] PATH_INFO: `/v3/projects/6d140898a8684238bddc9872a32eb94c` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:07.941815 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bedecd0-ba00-4081-90a2-4b491c9882c4 None admin] RBAC: Authorizing `identity:get_project(project_id=6d140898a8684238bddc9872a32eb94c)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:07.943542 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bedecd0-ba00-4081-90a2-4b491c9882c4 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:07.946555 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 113/231] 10.222.0.15 () {56 vars in 1186 bytes} [Sun Sep 11 10:33:07 2022] GET /identity/v3/projects/6d140898a8684238bddc9872a32eb94c => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:33:07.952161 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bedecd0-ba00-4081-90a2-4b491c9882c4 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:08.063218 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bedecd0-ba00-4081-90a2-4b491c9882c4 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:08.107222 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861695c8-e062-4c48-880f-a80792d8ae8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:08.108268 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861695c8-e062-4c48-880f-a80792d8ae8c None admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:08.108906 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861695c8-e062-4c48-880f-a80792d8ae8c None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:08.109402 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861695c8-e062-4c48-880f-a80792d8ae8c None admin] PATH_INFO: `/v3/projects/6d140898a8684238bddc9872a32eb94c/users/f5bed6f6065e483987356948f5be4b73/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:08.112037 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861695c8-e062-4c48-880f-a80792d8ae8c None admin] RBAC: Authorizing `identity:create_grant(project_id=6d140898a8684238bddc9872a32eb94c, user_id=f5bed6f6065e483987356948f5be4b73, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:08.114012 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861695c8-e062-4c48-880f-a80792d8ae8c None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:08.129778 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 114/232] 10.222.0.15 () {58 vars in 1439 bytes} [Sun Sep 11 10:33:07 2022] PUT /identity/v3/projects/6d140898a8684238bddc9872a32eb94c/users/f5bed6f6065e483987356948f5be4b73/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:33:08.194682 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c19fffb-7ba3-4c70-b9b1-c6b87a04077a None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:08.197933 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce04bdb-7864-4393-844b-7d39b3ac8ed8 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:08.198445 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce04bdb-7864-4393-844b-7d39b3ac8ed8 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:08.198912 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce04bdb-7864-4393-844b-7d39b3ac8ed8 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:08.200904 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 115/233] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:33:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:33:08.206304 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84a3e74-79a6-42cf-8212-e51951db7bf8 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:08.206919 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84a3e74-79a6-42cf-8212-e51951db7bf8 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:08.207384 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84a3e74-79a6-42cf-8212-e51951db7bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:08.227642 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-a84a3e74-79a6-42cf-8212-e51951db7bf8 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:33:08.229605 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-a84a3e74-79a6-42cf-8212-e51951db7bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:33:08.239999 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fb3df4-a552-448f-bcfd-0ce5936abaf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:08.240723 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fb3df4-a552-448f-bcfd-0ce5936abaf9 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:08.241004 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fb3df4-a552-448f-bcfd-0ce5936abaf9 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:08.241285 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fb3df4-a552-448f-bcfd-0ce5936abaf9 None admin] PATH_INFO: `/v3/projects/6d140898a8684238bddc9872a32eb94c` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:08.248311 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fb3df4-a552-448f-bcfd-0ce5936abaf9 None admin] RBAC: Authorizing `identity:get_project(project_id=6d140898a8684238bddc9872a32eb94c)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:08.249815 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fb3df4-a552-448f-bcfd-0ce5936abaf9 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:08.252113 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 119/234] 10.222.0.15 () {56 vars in 1186 bytes} [Sun Sep 11 10:33:07 2022] GET /identity/v3/projects/6d140898a8684238bddc9872a32eb94c => generated 302 bytes in 388 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:08.257885 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fb3df4-a552-448f-bcfd-0ce5936abaf9 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:08.263892 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:08.332014 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:08.350586 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 116/235] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:33:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:33:08.412587 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fb3df4-a552-448f-bcfd-0ce5936abaf9 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:08.437539 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72566c62-f26c-4c39-9197-2c682ced78bb None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:08.437835 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72566c62-f26c-4c39-9197-2c682ced78bb None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:08.438089 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72566c62-f26c-4c39-9197-2c682ced78bb None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:08.439453 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 117/236] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:33:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 10:33:08.452609 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38888595-a8dd-4524-b87d-8c4fb1c43683 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:08.452895 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38888595-a8dd-4524-b87d-8c4fb1c43683 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:08.453173 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38888595-a8dd-4524-b87d-8c4fb1c43683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:08.454409 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a078168-767a-4e1b-99b5-cd3126df3782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:08.455301 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a078168-767a-4e1b-99b5-cd3126df3782 None admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:08.455813 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a078168-767a-4e1b-99b5-cd3126df3782 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:08.456289 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a078168-767a-4e1b-99b5-cd3126df3782 None admin] PATH_INFO: `/v3/projects/6d140898a8684238bddc9872a32eb94c/users/f5bed6f6065e483987356948f5be4b73/roles/cdb3aa1bd85b4694964629ff7cf8039b` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:08.458699 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a078168-767a-4e1b-99b5-cd3126df3782 None admin] RBAC: Authorizing `identity:create_grant(project_id=6d140898a8684238bddc9872a32eb94c, user_id=f5bed6f6065e483987356948f5be4b73, role_id=cdb3aa1bd85b4694964629ff7cf8039b)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:08.460441 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a078168-767a-4e1b-99b5-cd3126df3782 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:08.480248 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 120/237] 10.222.0.15 () {58 vars in 1439 bytes} [Sun Sep 11 10:33:08 2022] PUT /identity/v3/projects/6d140898a8684238bddc9872a32eb94c/users/f5bed6f6065e483987356948f5be4b73/roles/cdb3aa1bd85b4694964629ff7cf8039b => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:33:08.488660 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-38888595-a8dd-4524-b87d-8c4fb1c43683 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:33:08.489956 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-38888595-a8dd-4524-b87d-8c4fb1c43683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:33:08.585923 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 118/238] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:33:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:33:08.734744 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a078168-767a-4e1b-99b5-cd3126df3782 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:08.835710 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:08.846891 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a078168-767a-4e1b-99b5-cd3126df3782 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:08.885338 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a3fb51-3c1d-4a8f-96f6-a09d9589ec16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:08.886049 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a3fb51-3c1d-4a8f-96f6-a09d9589ec16 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:08.886342 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a3fb51-3c1d-4a8f-96f6-a09d9589ec16 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:08.886703 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a3fb51-3c1d-4a8f-96f6-a09d9589ec16 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:08.902435 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a3fb51-3c1d-4a8f-96f6-a09d9589ec16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:33:08.907325 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-95a3fb51-3c1d-4a8f-96f6-a09d9589ec16 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 10:33:08.907325 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:08.907325 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:08.907325 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:08.907325 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:08.907325 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:08.907325 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:08.907325 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:08.907325 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:08.907325 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:08.907325 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:08.907325 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:08.907325 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:33:08.907325 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:33:08.907325 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:33:08.907325 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:33:08.908280 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:33:08.908280 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:33:08.908280 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:33:08.908280 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:33:08.908280 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:08.908280 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:08.908280 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:08.908280 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:08.908280 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:08.908280 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:08.908280 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:08.908280 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:08.908280 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:08.908280 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:08.908280 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:08.908280 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:08.909073 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:08.909073 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:08.909073 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:08.909073 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:08.909073 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:33:08.909073 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:33:08.909073 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:33:08.909073 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:33:08.909073 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:33:08.909073 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:33:08.909073 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 10:33:08.909073 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:33:08.909073 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 121/239] 10.222.0.15 () {56 vars in 1099 bytes} [Sun Sep 11 10:33:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:08.914351 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38888595-a8dd-4524-b87d-8c4fb1c43683 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:09.042482 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a3fb51-3c1d-4a8f-96f6-a09d9589ec16 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:09.048742 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:09.069619 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38888595-a8dd-4524-b87d-8c4fb1c43683 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:09.109543 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2805b82-a349-4c27-b706-86613b95784d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:09.110236 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2805b82-a349-4c27-b706-86613b95784d None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:09.110516 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2805b82-a349-4c27-b706-86613b95784d None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:09.110942 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2805b82-a349-4c27-b706-86613b95784d None admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:09.111639 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2805b82-a349-4c27-b706-86613b95784d None admin] RBAC: Adding query filter params (name=reader) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:09.111939 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2805b82-a349-4c27-b706-86613b95784d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:09.113612 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2805b82-a349-4c27-b706-86613b95784d None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:09.125974 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 119/240] 10.222.0.15 () {56 vars in 1101 bytes} [Sun Sep 11 10:33:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:09.136383 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2805b82-a349-4c27-b706-86613b95784d None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:09.205613 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a3fb51-3c1d-4a8f-96f6-a09d9589ec16 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:09.259413 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2805b82-a349-4c27-b706-86613b95784d None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:09.277233 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe4af13-eafb-4028-93b1-1faf566f99cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:09.278563 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe4af13-eafb-4028-93b1-1faf566f99cb None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:09.279046 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe4af13-eafb-4028-93b1-1faf566f99cb None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:09.279802 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe4af13-eafb-4028-93b1-1faf566f99cb None admin] PATH_INFO: `/v3/domains/default` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:09.281612 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe4af13-eafb-4028-93b1-1faf566f99cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:09.283934 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe4af13-eafb-4028-93b1-1faf566f99cb None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:09.286049 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 122/241] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:33:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:09.288196 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd8d25e-7bda-43d6-aec3-eddb04724c46 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:09.288821 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd8d25e-7bda-43d6-aec3-eddb04724c46 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:09.289270 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd8d25e-7bda-43d6-aec3-eddb04724c46 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:09.295853 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 123/242] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:33:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:33:09.296364 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd8d25e-7bda-43d6-aec3-eddb04724c46 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:09.305070 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95746be4-bee9-4b5f-a42b-e87c3f7630da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:09.306476 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95746be4-bee9-4b5f-a42b-e87c3f7630da None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:09.306765 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95746be4-bee9-4b5f-a42b-e87c3f7630da None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:09.307042 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95746be4-bee9-4b5f-a42b-e87c3f7630da None admin] PATH_INFO: `/v3/users/f6a0d29dab2243ee8115be5e732de396` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:09.327847 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95746be4-bee9-4b5f-a42b-e87c3f7630da None admin] RBAC: Authorizing `identity:get_user(user_id=f6a0d29dab2243ee8115be5e732de396)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:09.329317 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95746be4-bee9-4b5f-a42b-e87c3f7630da None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:09.331459 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 120/243] 10.222.0.15 () {56 vars in 1177 bytes} [Sun Sep 11 10:33:09 2022] GET /identity/v3/users/f6a0d29dab2243ee8115be5e732de396 => generated 297 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:09.337604 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf3cd50-0a54-4c0a-a8da-76292d60b986 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:09.337900 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf3cd50-0a54-4c0a-a8da-76292d60b986 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:09.337999 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:09.338176 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf3cd50-0a54-4c0a-a8da-76292d60b986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:09.345408 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:09.357463 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-daf3cd50-0a54-4c0a-a8da-76292d60b986 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:33:09.358788 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-daf3cd50-0a54-4c0a-a8da-76292d60b986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:33:09.428733 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd8d25e-7bda-43d6-aec3-eddb04724c46 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:09.436232 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 121/244] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:33:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:33:09.437195 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf3cd50-0a54-4c0a-a8da-76292d60b986 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:09.482535 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b67d483-df20-49bc-8741-b9af68153dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:09.484656 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b67d483-df20-49bc-8741-b9af68153dc5 None admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:09.485864 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b67d483-df20-49bc-8741-b9af68153dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:09.486280 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b67d483-df20-49bc-8741-b9af68153dc5 None admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:09.487314 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b67d483-df20-49bc-8741-b9af68153dc5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:09.491383 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b67d483-df20-49bc-8741-b9af68153dc5 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:09.529726 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 124/245] 10.222.0.15 () {60 vars in 1132 bytes} [Sun Sep 11 10:33:09 2022] POST /identity/v3/users => generated 293 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:33:09.606536 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf3cd50-0a54-4c0a-a8da-76292d60b986 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:09.662524 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6550c72-1548-4f26-baed-e09e0b904b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:09.663149 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6550c72-1548-4f26-baed-e09e0b904b2a None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:09.663443 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6550c72-1548-4f26-baed-e09e0b904b2a None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:09.663779 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6550c72-1548-4f26-baed-e09e0b904b2a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:09.666634 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:09.746526 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6550c72-1548-4f26-baed-e09e0b904b2a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:09.747959 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6550c72-1548-4f26-baed-e09e0b904b2a None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:09.843844 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b67d483-df20-49bc-8741-b9af68153dc5 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:09.876387 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 122/246] 10.222.0.15 () {58 vars in 1282 bytes} [Sun Sep 11 10:33:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 10:33:09.881934 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6550c72-1548-4f26-baed-e09e0b904b2a None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:09.922738 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:09.963814 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:09.991580 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b67d483-df20-49bc-8741-b9af68153dc5 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:10.004453 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6550c72-1548-4f26-baed-e09e0b904b2a None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:10.033352 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59514d0c-1eab-4976-ba52-e7c68974d693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:10.034450 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59514d0c-1eab-4976-ba52-e7c68974d693 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:10.034936 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59514d0c-1eab-4976-ba52-e7c68974d693 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:10.035888 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59514d0c-1eab-4976-ba52-e7c68974d693 None admin] PATH_INFO: `/v3/roles/member` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:10.049075 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59514d0c-1eab-4976-ba52-e7c68974d693 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:33:10.049871 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb58cc4-75e4-492d-abc5-e55419882ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:10.050522 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb58cc4-75e4-492d-abc5-e55419882ca9 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:10.050805 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb58cc4-75e4-492d-abc5-e55419882ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:10.051080 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb58cc4-75e4-492d-abc5-e55419882ca9 None admin] PATH_INFO: `/v3/projects/6d140898a8684238bddc9872a32eb94c` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:10.052254 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb58cc4-75e4-492d-abc5-e55419882ca9 None admin] RBAC: Authorizing `identity:get_project(project_id=6d140898a8684238bddc9872a32eb94c)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:10.053710 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb58cc4-75e4-492d-abc5-e55419882ca9 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:10.056615 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 123/247] 10.222.0.15 () {56 vars in 1186 bytes} [Sun Sep 11 10:33:09 2022] GET /identity/v3/projects/6d140898a8684238bddc9872a32eb94c => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:33:10.058948 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-59514d0c-1eab-4976-ba52-e7c68974d693 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 10:33:10.058948 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:10.058948 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:10.058948 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:10.058948 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:10.058948 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:10.058948 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:10.058948 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:10.058948 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:10.058948 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:10.058948 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:10.058948 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:10.058948 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:33:10.058948 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:33:10.058948 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:33:10.058948 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:33:10.059901 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:33:10.059901 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:33:10.059901 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:33:10.059901 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:33:10.059901 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:10.059901 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:10.059901 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:10.059901 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:10.059901 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:10.059901 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:10.059901 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:10.059901 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:10.059901 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:10.059901 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:10.059901 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:10.059901 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:10.064867 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:10.064867 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:10.064867 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:10.064867 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:10.064867 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:33:10.064867 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:33:10.064867 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:33:10.064867 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:33:10.064867 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:33:10.064867 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:33:10.064867 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 10:33:10.064867 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:33:10.064867 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 125/248] 10.222.0.15 () {56 vars in 1099 bytes} [Sun Sep 11 10:33:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:10.065540 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb58cc4-75e4-492d-abc5-e55419882ca9 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:10.067511 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59514d0c-1eab-4976-ba52-e7c68974d693 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:10.194788 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb58cc4-75e4-492d-abc5-e55419882ca9 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:10.196416 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:10.209133 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59514d0c-1eab-4976-ba52-e7c68974d693 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:10.241775 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b607af-f381-47e2-b6e8-6c093d9acf66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:10.242405 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b607af-f381-47e2-b6e8-6c093d9acf66 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:10.243312 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b607af-f381-47e2-b6e8-6c093d9acf66 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:10.243712 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b607af-f381-47e2-b6e8-6c093d9acf66 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:10.244447 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b607af-f381-47e2-b6e8-6c093d9acf66 None admin] RBAC: Adding query filter params (role.id=d577f4ba2f9c412e9530e50d7f3d6bd7, scope.project.id=6d140898a8684238bddc9872a32eb94c, user.id=f6a0d29dab2243ee8115be5e732de396) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:10.245086 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b607af-f381-47e2-b6e8-6c093d9acf66 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:10.246678 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b607af-f381-47e2-b6e8-6c093d9acf66 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:10.255655 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 124/249] 10.222.0.15 () {56 vars in 1374 bytes} [Sun Sep 11 10:33:10 2022] GET /identity/v3/role_assignments?user.id=f6a0d29dab2243ee8115be5e732de396&scope.project.id=6d140898a8684238bddc9872a32eb94c&role.id=d577f4ba2f9c412e9530e50d7f3d6bd7 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:33:10.264987 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ab6829-f93a-46cf-9273-00a2fe9f405f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:10.265731 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ab6829-f93a-46cf-9273-00a2fe9f405f None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:10.266026 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ab6829-f93a-46cf-9273-00a2fe9f405f None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:10.266300 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ab6829-f93a-46cf-9273-00a2fe9f405f None admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:10.266992 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ab6829-f93a-46cf-9273-00a2fe9f405f None admin] RBAC: Adding query filter params (name=member) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:10.267297 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ab6829-f93a-46cf-9273-00a2fe9f405f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:10.268827 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ab6829-f93a-46cf-9273-00a2fe9f405f None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:10.271640 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:10.280903 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 126/250] 10.222.0.15 () {56 vars in 1101 bytes} [Sun Sep 11 10:33:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:10.286111 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b607af-f381-47e2-b6e8-6c093d9acf66 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:10.420455 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b607af-f381-47e2-b6e8-6c093d9acf66 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:10.471685 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f69107-b12f-4b80-a714-c8a9808f6891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:10.472961 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f69107-b12f-4b80-a714-c8a9808f6891 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:10.473437 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f69107-b12f-4b80-a714-c8a9808f6891 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:10.473899 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f69107-b12f-4b80-a714-c8a9808f6891 None admin] PATH_INFO: `/v3/users/5e1ca49806f84e66a9cc647892bcea5c` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:10.475244 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f69107-b12f-4b80-a714-c8a9808f6891 None admin] RBAC: Authorizing `identity:get_user(user_id=5e1ca49806f84e66a9cc647892bcea5c)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:10.477807 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f69107-b12f-4b80-a714-c8a9808f6891 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:10.480962 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 125/251] 10.222.0.15 () {56 vars in 1177 bytes} [Sun Sep 11 10:33:10 2022] GET /identity/v3/users/5e1ca49806f84e66a9cc647892bcea5c => generated 297 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:33:10.485533 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ab6829-f93a-46cf-9273-00a2fe9f405f None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:10.641033 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ab6829-f93a-46cf-9273-00a2fe9f405f None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:10.682600 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df36a7f-560a-44e5-8959-2ddd76295476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:10.683249 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df36a7f-560a-44e5-8959-2ddd76295476 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:10.683603 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df36a7f-560a-44e5-8959-2ddd76295476 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:10.683884 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df36a7f-560a-44e5-8959-2ddd76295476 None admin] PATH_INFO: `/v3/projects/6d140898a8684238bddc9872a32eb94c` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:10.685027 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df36a7f-560a-44e5-8959-2ddd76295476 None admin] RBAC: Authorizing `identity:get_project(project_id=6d140898a8684238bddc9872a32eb94c)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:10.686488 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df36a7f-560a-44e5-8959-2ddd76295476 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:10.688474 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 127/252] 10.222.0.15 () {56 vars in 1186 bytes} [Sun Sep 11 10:33:10 2022] GET /identity/v3/projects/6d140898a8684238bddc9872a32eb94c => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:10.693796 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f69107-b12f-4b80-a714-c8a9808f6891 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:10.746419 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:10.820617 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f69107-b12f-4b80-a714-c8a9808f6891 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:10.866259 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444245b6-c681-4385-8955-29065bd3e8a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:10.867303 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444245b6-c681-4385-8955-29065bd3e8a8 None admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:10.868458 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444245b6-c681-4385-8955-29065bd3e8a8 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:10.869165 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444245b6-c681-4385-8955-29065bd3e8a8 None admin] PATH_INFO: `/v3/projects/6d140898a8684238bddc9872a32eb94c/users/5e1ca49806f84e66a9cc647892bcea5c/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:10.877406 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444245b6-c681-4385-8955-29065bd3e8a8 None admin] RBAC: Authorizing `identity:create_grant(project_id=6d140898a8684238bddc9872a32eb94c, user_id=5e1ca49806f84e66a9cc647892bcea5c, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:10.879117 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444245b6-c681-4385-8955-29065bd3e8a8 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:10.899588 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 126/253] 10.222.0.15 () {58 vars in 1439 bytes} [Sun Sep 11 10:33:10 2022] PUT /identity/v3/projects/6d140898a8684238bddc9872a32eb94c/users/5e1ca49806f84e66a9cc647892bcea5c/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:33:11.010757 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862655a5-c96d-45c9-b1e6-dec854164009 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:11.011457 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862655a5-c96d-45c9-b1e6-dec854164009 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:11.011927 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862655a5-c96d-45c9-b1e6-dec854164009 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:11.013496 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 128/254] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:33:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:33:11.033505 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a72260-d7d6-49d7-8a17-1127a7229871 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:11.034117 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a72260-d7d6-49d7-8a17-1127a7229871 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:11.034610 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a72260-d7d6-49d7-8a17-1127a7229871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:11.063138 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-c5a72260-d7d6-49d7-8a17-1127a7229871 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:33:11.064693 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-c5a72260-d7d6-49d7-8a17-1127a7229871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:33:11.154353 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 127/255] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:33:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:33:11.296023 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11593328-7d96-4d53-bcfb-f3195b7eb651 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:11.296979 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11593328-7d96-4d53-bcfb-f3195b7eb651 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:11.297444 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11593328-7d96-4d53-bcfb-f3195b7eb651 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:11.299254 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 129/256] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:33:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:33:11.304435 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f712416-8b12-48b3-863b-47ef5ab02fa1 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:11.305342 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f712416-8b12-48b3-863b-47ef5ab02fa1 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:11.306107 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f712416-8b12-48b3-863b-47ef5ab02fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:11.332472 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-1f712416-8b12-48b3-863b-47ef5ab02fa1 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:33:11.333994 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-1f712416-8b12-48b3-863b-47ef5ab02fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:33:11.355277 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:11.422725 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 128/257] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:33:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:33:11.644754 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11593328-7d96-4d53-bcfb-f3195b7eb651 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:11.735388 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:11.779948 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11593328-7d96-4d53-bcfb-f3195b7eb651 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:11.815190 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee83670-f5c9-4a78-921c-26f1b5fc3762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:11.815845 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee83670-f5c9-4a78-921c-26f1b5fc3762 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:11.816120 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee83670-f5c9-4a78-921c-26f1b5fc3762 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:11.816394 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee83670-f5c9-4a78-921c-26f1b5fc3762 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:11.827470 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee83670-f5c9-4a78-921c-26f1b5fc3762 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:33:11.832443 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-fee83670-f5c9-4a78-921c-26f1b5fc3762 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 10:33:11.832443 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:11.832443 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:11.832443 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:11.832443 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:11.832443 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:11.832443 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:11.832443 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:11.832443 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:11.832443 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:11.832443 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:11.832443 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:11.832443 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:33:11.832443 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:33:11.832443 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:33:11.832443 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:33:11.833331 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:33:11.833331 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:33:11.833331 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:33:11.833331 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:33:11.833331 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:11.833331 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:11.833331 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:11.833331 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:11.833331 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:11.833331 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:11.833331 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:11.833331 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:11.833331 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:11.833331 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:11.833331 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:11.833331 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:11.834127 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:11.834127 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:11.834127 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:11.834127 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:11.834127 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:33:11.834127 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:33:11.834127 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:33:11.834127 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:33:11.834127 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:33:11.834127 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:33:11.834127 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 10:33:11.834127 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:33:11.834127 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 130/258] 10.222.0.15 () {56 vars in 1096 bytes} [Sun Sep 11 10:33:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:11.840744 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f712416-8b12-48b3-863b-47ef5ab02fa1 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:11.897007 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee83670-f5c9-4a78-921c-26f1b5fc3762 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:11.964532 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f712416-8b12-48b3-863b-47ef5ab02fa1 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:12.052968 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5acfe17-a2ef-4cbe-bd01-67b04aeeb60b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:12.054092 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5acfe17-a2ef-4cbe-bd01-67b04aeeb60b None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:12.060676 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5acfe17-a2ef-4cbe-bd01-67b04aeeb60b None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:12.061386 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5acfe17-a2ef-4cbe-bd01-67b04aeeb60b None admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:12.063502 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5acfe17-a2ef-4cbe-bd01-67b04aeeb60b None admin] RBAC: Adding query filter params (name=admin) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:12.065008 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5acfe17-a2ef-4cbe-bd01-67b04aeeb60b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:12.066646 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5acfe17-a2ef-4cbe-bd01-67b04aeeb60b None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:12.091002 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 129/259] 10.222.0.15 () {56 vars in 1099 bytes} [Sun Sep 11 10:33:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:12.104685 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee83670-f5c9-4a78-921c-26f1b5fc3762 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:12.105907 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5acfe17-a2ef-4cbe-bd01-67b04aeeb60b None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:12.147771 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:12.167855 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:12.192682 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e1c1e3-3514-483f-b1ca-c1c52d18e918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:12.193336 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e1c1e3-3514-483f-b1ca-c1c52d18e918 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:12.193618 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e1c1e3-3514-483f-b1ca-c1c52d18e918 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:12.193894 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e1c1e3-3514-483f-b1ca-c1c52d18e918 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:12.216770 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e1c1e3-3514-483f-b1ca-c1c52d18e918 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:33:12.221619 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-d0e1c1e3-3514-483f-b1ca-c1c52d18e918 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 10:33:12.221619 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:12.221619 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:12.221619 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:12.221619 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:12.221619 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:12.221619 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:12.221619 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:12.221619 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:12.221619 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:12.221619 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:12.221619 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:12.221619 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:33:12.221619 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:33:12.221619 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:33:12.221619 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:33:12.222590 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:33:12.222590 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:33:12.222590 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:33:12.222590 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:33:12.222590 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:12.222590 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:12.222590 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:12.222590 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:12.222590 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:12.222590 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:12.222590 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:12.222590 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:12.222590 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:12.222590 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:12.222590 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:12.222590 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:12.223410 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:12.223410 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:12.223410 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:12.223410 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:12.223410 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:33:12.223410 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:33:12.223410 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:33:12.223410 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:33:12.223410 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:33:12.223410 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:33:12.223410 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 10:33:12.223410 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:33:12.223410 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 131/260] 10.222.0.15 () {56 vars in 1114 bytes} [Sun Sep 11 10:33:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 10:33:12.232402 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e1c1e3-3514-483f-b1ca-c1c52d18e918 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:12.344804 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5acfe17-a2ef-4cbe-bd01-67b04aeeb60b None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:12.409844 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4f2867-0345-48b3-b4c9-7e554c6c0ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:12.410488 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4f2867-0345-48b3-b4c9-7e554c6c0ea4 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:12.410769 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4f2867-0345-48b3-b4c9-7e554c6c0ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:12.411045 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4f2867-0345-48b3-b4c9-7e554c6c0ea4 None admin] PATH_INFO: `/v3/users/f5bed6f6065e483987356948f5be4b73` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:12.412270 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4f2867-0345-48b3-b4c9-7e554c6c0ea4 None admin] RBAC: Authorizing `identity:get_user(user_id=f5bed6f6065e483987356948f5be4b73)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:12.413710 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4f2867-0345-48b3-b4c9-7e554c6c0ea4 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:12.415694 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 130/261] 10.222.0.15 () {56 vars in 1177 bytes} [Sun Sep 11 10:33:12 2022] GET /identity/v3/users/f5bed6f6065e483987356948f5be4b73 => generated 283 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:33:12.421653 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4f2867-0345-48b3-b4c9-7e554c6c0ea4 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:12.475347 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e1c1e3-3514-483f-b1ca-c1c52d18e918 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:12.565150 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e896fd0e-ec76-4d20-88e5-8550f2388547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:12.566109 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e896fd0e-ec76-4d20-88e5-8550f2388547 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:12.566585 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e896fd0e-ec76-4d20-88e5-8550f2388547 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:12.567047 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e896fd0e-ec76-4d20-88e5-8550f2388547 None admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:12.568037 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e896fd0e-ec76-4d20-88e5-8550f2388547 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:12.568561 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e896fd0e-ec76-4d20-88e5-8550f2388547 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:12.570132 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e896fd0e-ec76-4d20-88e5-8550f2388547 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:12.588803 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 132/262] 10.222.0.15 () {56 vars in 1111 bytes} [Sun Sep 11 10:33:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:12.594870 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e896fd0e-ec76-4d20-88e5-8550f2388547 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:12.602413 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:12.634303 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4f2867-0345-48b3-b4c9-7e554c6c0ea4 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:12.708161 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba43e1e-3f65-4ef9-b0af-1b2f5f692d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:12.709627 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba43e1e-3f65-4ef9-b0af-1b2f5f692d2b None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:12.710121 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba43e1e-3f65-4ef9-b0af-1b2f5f692d2b None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:12.710589 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba43e1e-3f65-4ef9-b0af-1b2f5f692d2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:12.715702 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:12.807228 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e896fd0e-ec76-4d20-88e5-8550f2388547 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:12.848445 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba43e1e-3f65-4ef9-b0af-1b2f5f692d2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:12.849833 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba43e1e-3f65-4ef9-b0af-1b2f5f692d2b None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:12.873260 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74488cb9-af18-4931-9f75-b9b1fcaf1717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:12.875268 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74488cb9-af18-4931-9f75-b9b1fcaf1717 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:12.876125 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74488cb9-af18-4931-9f75-b9b1fcaf1717 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:12.876400 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74488cb9-af18-4931-9f75-b9b1fcaf1717 None admin] PATH_INFO: `/v3/users/f5bed6f6065e483987356948f5be4b73` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:12.878326 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74488cb9-af18-4931-9f75-b9b1fcaf1717 None admin] RBAC: Authorizing `identity:get_user(user_id=f5bed6f6065e483987356948f5be4b73)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:12.880592 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74488cb9-af18-4931-9f75-b9b1fcaf1717 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:12.882171 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 133/263] 10.222.0.15 () {56 vars in 1177 bytes} [Sun Sep 11 10:33:12 2022] GET /identity/v3/users/f5bed6f6065e483987356948f5be4b73 => generated 283 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:12.900265 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74488cb9-af18-4931-9f75-b9b1fcaf1717 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:13.021071 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:13.061612 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 131/264] 10.222.0.15 () {58 vars in 1282 bytes} [Sun Sep 11 10:33:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 641 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 11 10:33:13.067472 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba43e1e-3f65-4ef9-b0af-1b2f5f692d2b None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:13.091179 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74488cb9-af18-4931-9f75-b9b1fcaf1717 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:13.123613 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:13.160334 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4817793a-7d95-404f-9b3a-44a69610323b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:13.161407 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4817793a-7d95-404f-9b3a-44a69610323b None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:13.161690 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4817793a-7d95-404f-9b3a-44a69610323b None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:13.161963 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4817793a-7d95-404f-9b3a-44a69610323b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:13.272412 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba43e1e-3f65-4ef9-b0af-1b2f5f692d2b None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:13.304434 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4817793a-7d95-404f-9b3a-44a69610323b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:13.311612 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4817793a-7d95-404f-9b3a-44a69610323b None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:13.331595 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63c8c02-ce86-4809-8f97-eda81605b50a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:13.332431 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63c8c02-ce86-4809-8f97-eda81605b50a None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:13.332716 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63c8c02-ce86-4809-8f97-eda81605b50a None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:13.332984 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63c8c02-ce86-4809-8f97-eda81605b50a None admin] PATH_INFO: `/v3/projects/6d140898a8684238bddc9872a32eb94c` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:13.334105 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63c8c02-ce86-4809-8f97-eda81605b50a None admin] RBAC: Authorizing `identity:get_project(project_id=6d140898a8684238bddc9872a32eb94c)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:13.335578 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63c8c02-ce86-4809-8f97-eda81605b50a None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:13.339018 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 132/265] 10.222.0.15 () {56 vars in 1186 bytes} [Sun Sep 11 10:33:13 2022] GET /identity/v3/projects/6d140898a8684238bddc9872a32eb94c => generated 302 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:33:13.340456 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba46080e-4a94-48f1-be9c-1752bbf21465 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:13.340733 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba46080e-4a94-48f1-be9c-1752bbf21465 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:13.340999 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba46080e-4a94-48f1-be9c-1752bbf21465 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:13.342196 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 133/266] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:33:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:33:13.347117 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba46080e-4a94-48f1-be9c-1752bbf21465 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:13.415043 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:13.453438 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 134/267] 10.222.0.15 () {58 vars in 1282 bytes} [Sun Sep 11 10:33:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 549 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 10:33:13.454783 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b95a44-1e66-424f-81c4-7c685e9d4e5e None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:13.455068 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b95a44-1e66-424f-81c4-7c685e9d4e5e None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:13.455342 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b95a44-1e66-424f-81c4-7c685e9d4e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:13.471244 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba46080e-4a94-48f1-be9c-1752bbf21465 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:13.475082 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-a5b95a44-1e66-424f-81c4-7c685e9d4e5e None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:33:13.476301 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-a5b95a44-1e66-424f-81c4-7c685e9d4e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:33:13.492203 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:13.517858 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90648ff7-6fb8-4e85-8e00-bd2227a28165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:13.518864 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90648ff7-6fb8-4e85-8e00-bd2227a28165 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:13.519410 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90648ff7-6fb8-4e85-8e00-bd2227a28165 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:13.519895 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90648ff7-6fb8-4e85-8e00-bd2227a28165 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:13.520923 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90648ff7-6fb8-4e85-8e00-bd2227a28165 None admin] RBAC: Adding query filter params (role.id=2cb7d73f0ea8414296367573f6902340, scope.project.id=6d140898a8684238bddc9872a32eb94c, user.id=f5bed6f6065e483987356948f5be4b73) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:13.521458 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90648ff7-6fb8-4e85-8e00-bd2227a28165 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:13.523192 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90648ff7-6fb8-4e85-8e00-bd2227a28165 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:13.530163 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 134/268] 10.222.0.15 () {56 vars in 1374 bytes} [Sun Sep 11 10:33:13 2022] GET /identity/v3/role_assignments?user.id=f5bed6f6065e483987356948f5be4b73&scope.project.id=6d140898a8684238bddc9872a32eb94c&role.id=2cb7d73f0ea8414296367573f6902340 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:13.537163 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14be42d4-9584-465f-8dd2-729cc6484831 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:13.537877 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14be42d4-9584-465f-8dd2-729cc6484831 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:13.538413 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14be42d4-9584-465f-8dd2-729cc6484831 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:13.548664 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 135/269] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:33:13 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:33:13.550410 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14be42d4-9584-465f-8dd2-729cc6484831 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:13.575019 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 135/270] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:33:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:33:13.576503 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c638953-b5f8-460b-a31b-f5dc7710a595 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:13.576794 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c638953-b5f8-460b-a31b-f5dc7710a595 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:13.577067 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c638953-b5f8-460b-a31b-f5dc7710a595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:13.613417 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-5c638953-b5f8-460b-a31b-f5dc7710a595 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:33:13.614719 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-5c638953-b5f8-460b-a31b-f5dc7710a595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:33:13.754990 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 136/271] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:33:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:33:13.756238 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a057b13a-80ef-43a5-a6ba-4988ac003d1f None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:13.756523 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a057b13a-80ef-43a5-a6ba-4988ac003d1f None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:13.756800 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a057b13a-80ef-43a5-a6ba-4988ac003d1f None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:13.758817 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 137/272] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:33:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:33:13.765184 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd765833-80a3-47c0-a7e1-2a72364e97d5 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:13.765473 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd765833-80a3-47c0-a7e1-2a72364e97d5 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:13.765751 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd765833-80a3-47c0-a7e1-2a72364e97d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:13.766105 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14be42d4-9584-465f-8dd2-729cc6484831 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:13.784800 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:13.801736 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-fd765833-80a3-47c0-a7e1-2a72364e97d5 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:33:13.802922 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-fd765833-80a3-47c0-a7e1-2a72364e97d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:33:13.805698 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5348a9-0549-415e-abca-8f32f82982cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:13.807017 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5348a9-0549-415e-abca-8f32f82982cb None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:13.807301 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5348a9-0549-415e-abca-8f32f82982cb None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:13.807601 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5348a9-0549-415e-abca-8f32f82982cb None admin] PATH_INFO: `/v3/projects/6d140898a8684238bddc9872a32eb94c` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:13.808777 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5348a9-0549-415e-abca-8f32f82982cb None admin] RBAC: Authorizing `identity:get_project(project_id=6d140898a8684238bddc9872a32eb94c)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:13.810268 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5348a9-0549-415e-abca-8f32f82982cb None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:13.817026 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 136/273] 10.222.0.15 () {56 vars in 1186 bytes} [Sun Sep 11 10:33:13 2022] GET /identity/v3/projects/6d140898a8684238bddc9872a32eb94c => generated 302 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:13.818203 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5348a9-0549-415e-abca-8f32f82982cb None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:13.925192 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 138/274] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:33:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:33:13.952450 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5348a9-0549-415e-abca-8f32f82982cb None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:13.959131 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd765833-80a3-47c0-a7e1-2a72364e97d5 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:13.970383 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:14.019323 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5f89d5-4a79-4e4e-bda7-292570785ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:14.019323 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5f89d5-4a79-4e4e-bda7-292570785ba9 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:14.019323 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5f89d5-4a79-4e4e-bda7-292570785ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:14.019323 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5f89d5-4a79-4e4e-bda7-292570785ba9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:14.019943 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5f89d5-4a79-4e4e-bda7-292570785ba9 None admin] RBAC: Adding query filter params (role.id=cdb3aa1bd85b4694964629ff7cf8039b, scope.project.id=6d140898a8684238bddc9872a32eb94c, user.id=f5bed6f6065e483987356948f5be4b73) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:14.019943 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5f89d5-4a79-4e4e-bda7-292570785ba9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:14.019943 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5f89d5-4a79-4e4e-bda7-292570785ba9 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:14.025824 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 137/275] 10.222.0.15 () {56 vars in 1374 bytes} [Sun Sep 11 10:33:13 2022] GET /identity/v3/role_assignments?user.id=f5bed6f6065e483987356948f5be4b73&scope.project.id=6d140898a8684238bddc9872a32eb94c&role.id=cdb3aa1bd85b4694964629ff7cf8039b => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:14.067303 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5f89d5-4a79-4e4e-bda7-292570785ba9 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:14.106078 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd765833-80a3-47c0-a7e1-2a72364e97d5 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:14.153822 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7859a0-55cc-42f7-a445-10384e09715d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:14.154779 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7859a0-55cc-42f7-a445-10384e09715d None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:14.155254 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7859a0-55cc-42f7-a445-10384e09715d None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:14.155719 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7859a0-55cc-42f7-a445-10384e09715d None admin] PATH_INFO: `/v3/domains/default` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:14.162032 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7859a0-55cc-42f7-a445-10384e09715d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:14.163896 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7859a0-55cc-42f7-a445-10384e09715d None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:14.165872 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 139/276] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:33:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:33:14.168653 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:14.172148 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7859a0-55cc-42f7-a445-10384e09715d None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:14.225034 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5f89d5-4a79-4e4e-bda7-292570785ba9 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:14.264567 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76b6b0f-8788-4b8b-9687-3b0ffd247d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:14.265227 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76b6b0f-8788-4b8b-9687-3b0ffd247d3d None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:14.265608 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76b6b0f-8788-4b8b-9687-3b0ffd247d3d None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:14.265882 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76b6b0f-8788-4b8b-9687-3b0ffd247d3d None admin] PATH_INFO: `/v3/roles/member` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:14.278588 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:14.280199 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76b6b0f-8788-4b8b-9687-3b0ffd247d3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:33:14.293442 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-a76b6b0f-8788-4b8b-9687-3b0ffd247d3d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 10:33:14.293442 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:14.293442 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:14.293442 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:14.293442 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:14.293442 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:14.293442 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:14.293442 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:14.293442 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:14.293442 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:14.293442 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:14.293442 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:14.293442 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:33:14.293442 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:33:14.293442 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:33:14.293442 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:33:14.294519 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:33:14.294519 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:33:14.294519 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:33:14.294519 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:33:14.294519 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:14.294519 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:14.294519 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:14.294519 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:14.294519 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:14.294519 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:14.294519 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:14.294519 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:14.294519 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:14.294519 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:14.294519 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:14.294519 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:14.295316 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:14.295316 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:14.295316 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:14.295316 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:14.295316 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:33:14.295316 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:33:14.295316 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:33:14.295316 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:33:14.295316 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:33:14.295316 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:33:14.295316 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 10:33:14.295316 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:33:14.295316 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 138/277] 10.222.0.15 () {56 vars in 1099 bytes} [Sun Sep 11 10:33:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 10:33:14.300531 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76b6b0f-8788-4b8b-9687-3b0ffd247d3d None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:14.311139 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7859a0-55cc-42f7-a445-10384e09715d None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:14.359378 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2cb411-ee9e-4567-aeb8-c8874403bd68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:14.360058 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2cb411-ee9e-4567-aeb8-c8874403bd68 None admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:14.360340 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2cb411-ee9e-4567-aeb8-c8874403bd68 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:14.360614 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2cb411-ee9e-4567-aeb8-c8874403bd68 None admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:14.361702 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2cb411-ee9e-4567-aeb8-c8874403bd68 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:14.363469 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2cb411-ee9e-4567-aeb8-c8874403bd68 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:14.395732 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 140/278] 10.222.0.15 () {60 vars in 1132 bytes} [Sun Sep 11 10:33:14 2022] POST /identity/v3/users => generated 293 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 10:33:14.396571 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2cb411-ee9e-4567-aeb8-c8874403bd68 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:14.437923 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76b6b0f-8788-4b8b-9687-3b0ffd247d3d None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:14.456240 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:14.495560 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4a97be-3580-4f58-8ae3-3519e963dcac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:14.496182 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a97be-3580-4f58-8ae3-3519e963dcac None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:14.496461 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a97be-3580-4f58-8ae3-3519e963dcac None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:14.496736 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a97be-3580-4f58-8ae3-3519e963dcac None admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:14.497418 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4a97be-3580-4f58-8ae3-3519e963dcac None admin] RBAC: Adding query filter params (name=member) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:14.497723 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4a97be-3580-4f58-8ae3-3519e963dcac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:14.499236 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4a97be-3580-4f58-8ae3-3519e963dcac None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:14.514563 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 139/279] 10.222.0.15 () {56 vars in 1101 bytes} [Sun Sep 11 10:33:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:33:14.514563 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b4d66f-8b6c-413c-83aa-501b3ea450e7 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:14.514563 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b4d66f-8b6c-413c-83aa-501b3ea450e7 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:14.514563 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b4d66f-8b6c-413c-83aa-501b3ea450e7 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:14.514563 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 140/280] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:33:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:33:14.518630 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b4d66f-8b6c-413c-83aa-501b3ea450e7 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:14.529505 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2cb411-ee9e-4567-aeb8-c8874403bd68 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:14.560606 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:14.568383 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2e7468-1eba-4de1-ad6e-072b3d76f5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:14.569057 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2e7468-1eba-4de1-ad6e-072b3d76f5f7 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:14.569344 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2e7468-1eba-4de1-ad6e-072b3d76f5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:14.569897 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2e7468-1eba-4de1-ad6e-072b3d76f5f7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:14.581722 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2e7468-1eba-4de1-ad6e-072b3d76f5f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:33:14.587625 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-4c2e7468-1eba-4de1-ad6e-072b3d76f5f7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 10:33:14.587625 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:14.587625 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:14.587625 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:14.587625 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:14.587625 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:14.587625 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:14.587625 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:14.587625 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:14.587625 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:14.587625 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:14.587625 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:14.587625 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:33:14.587625 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:33:14.587625 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:33:14.587625 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:33:14.588565 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:33:14.588565 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:33:14.588565 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:33:14.588565 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:33:14.588565 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:14.588565 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:14.588565 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:14.588565 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:14.588565 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:14.588565 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:14.588565 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:14.588565 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:14.588565 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:14.588565 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:14.588565 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:14.588565 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:14.589402 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:14.589402 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:14.589402 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:14.589402 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:14.589402 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:33:14.589402 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:33:14.589402 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:33:14.589402 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:33:14.589402 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:33:14.589402 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:33:14.589402 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 10:33:14.589402 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:33:14.589402 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 141/281] 10.222.0.15 () {56 vars in 1099 bytes} [Sun Sep 11 10:33:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:14.590074 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae9ac4a-0651-4a55-bdd1-de39126c3272 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:14.590187 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae9ac4a-0651-4a55-bdd1-de39126c3272 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:14.591164 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae9ac4a-0651-4a55-bdd1-de39126c3272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:14.612140 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-9ae9ac4a-0651-4a55-bdd1-de39126c3272 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:33:14.613331 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-9ae9ac4a-0651-4a55-bdd1-de39126c3272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:33:14.648155 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b4d66f-8b6c-413c-83aa-501b3ea450e7 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:14.695052 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c9b771-7d75-47c1-8e12-cd2192116f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:14.695949 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c9b771-7d75-47c1-8e12-cd2192116f77 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:14.696221 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c9b771-7d75-47c1-8e12-cd2192116f77 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:14.696493 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c9b771-7d75-47c1-8e12-cd2192116f77 None admin] PATH_INFO: `/v3/users/60b9fd46b70244c4a289ae23c5e407fb` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:14.696903 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 142/282] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:33:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:33:14.697627 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae9ac4a-0651-4a55-bdd1-de39126c3272 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:14.715393 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c9b771-7d75-47c1-8e12-cd2192116f77 None admin] RBAC: Authorizing `identity:get_user(user_id=60b9fd46b70244c4a289ae23c5e407fb)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:14.716957 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c9b771-7d75-47c1-8e12-cd2192116f77 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:14.720916 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 141/283] 10.222.0.15 () {56 vars in 1177 bytes} [Sun Sep 11 10:33:14 2022] GET /identity/v3/users/60b9fd46b70244c4a289ae23c5e407fb => generated 293 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:14.724707 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c9b771-7d75-47c1-8e12-cd2192116f77 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:14.757564 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:14.825189 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae9ac4a-0651-4a55-bdd1-de39126c3272 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:14.848115 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c9b771-7d75-47c1-8e12-cd2192116f77 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:14.853824 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:14.874110 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ede0656-4ae4-4d52-ab50-f5bbe5b88552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:14.875170 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ede0656-4ae4-4d52-ab50-f5bbe5b88552 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:14.875791 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ede0656-4ae4-4d52-ab50-f5bbe5b88552 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:14.876262 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ede0656-4ae4-4d52-ab50-f5bbe5b88552 None admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:14.877125 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ede0656-4ae4-4d52-ab50-f5bbe5b88552 None admin] RBAC: Adding query filter params (name=reader) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:14.877620 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ede0656-4ae4-4d52-ab50-f5bbe5b88552 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:14.879218 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ede0656-4ae4-4d52-ab50-f5bbe5b88552 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:14.891965 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 143/284] 10.222.0.15 () {56 vars in 1101 bytes} [Sun Sep 11 10:33:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:14.900746 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ede0656-4ae4-4d52-ab50-f5bbe5b88552 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:14.903042 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcba132-7a94-4a24-a781-b1d2842fc3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:14.903921 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcba132-7a94-4a24-a781-b1d2842fc3df None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:14.904493 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcba132-7a94-4a24-a781-b1d2842fc3df None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:14.904985 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcba132-7a94-4a24-a781-b1d2842fc3df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:14.906010 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcba132-7a94-4a24-a781-b1d2842fc3df None admin] RBAC: Adding query filter params (role.id=f836487a66c74c05838133c9d0614bbf, user.id=60b9fd46b70244c4a289ae23c5e407fb, scope.system=all) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:14.906534 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcba132-7a94-4a24-a781-b1d2842fc3df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:14.908241 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcba132-7a94-4a24-a781-b1d2842fc3df None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:14.927844 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 142/285] 10.222.0.15 () {56 vars in 1308 bytes} [Sun Sep 11 10:33:14 2022] GET /identity/v3/role_assignments?user.id=60b9fd46b70244c4a289ae23c5e407fb&scope.system=all&role.id=f836487a66c74c05838133c9d0614bbf => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:33:14.987768 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcba132-7a94-4a24-a781-b1d2842fc3df None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:15.092098 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ede0656-4ae4-4d52-ab50-f5bbe5b88552 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:15.141905 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:15.152987 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcba132-7a94-4a24-a781-b1d2842fc3df None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:15.185545 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ac01e7-4e60-4e44-80b1-f6c29bd2ebef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:15.186196 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ac01e7-4e60-4e44-80b1-f6c29bd2ebef None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:15.186479 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ac01e7-4e60-4e44-80b1-f6c29bd2ebef None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:15.186743 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ac01e7-4e60-4e44-80b1-f6c29bd2ebef None admin] PATH_INFO: `/v3/users/f6a0d29dab2243ee8115be5e732de396` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:15.189760 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ac01e7-4e60-4e44-80b1-f6c29bd2ebef None admin] RBAC: Authorizing `identity:get_user(user_id=f6a0d29dab2243ee8115be5e732de396)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:15.191482 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ac01e7-4e60-4e44-80b1-f6c29bd2ebef None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:15.191981 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be55f0b-79e1-409b-aaf9-1a92bf1b0694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:15.192717 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be55f0b-79e1-409b-aaf9-1a92bf1b0694 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:15.193090 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be55f0b-79e1-409b-aaf9-1a92bf1b0694 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:15.193449 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be55f0b-79e1-409b-aaf9-1a92bf1b0694 None admin] PATH_INFO: `/v3/roles/member` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:15.193569 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 144/286] 10.222.0.15 () {56 vars in 1177 bytes} [Sun Sep 11 10:33:14 2022] GET /identity/v3/users/f6a0d29dab2243ee8115be5e732de396 => generated 297 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:15.200201 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ac01e7-4e60-4e44-80b1-f6c29bd2ebef None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:15.207195 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be55f0b-79e1-409b-aaf9-1a92bf1b0694 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:33:15.212366 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-2be55f0b-79e1-409b-aaf9-1a92bf1b0694 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 10:33:15.212366 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:15.212366 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:15.212366 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:15.212366 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:15.212366 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:15.212366 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:15.212366 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:15.212366 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:15.212366 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:15.212366 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:15.212366 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:15.212366 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:33:15.212366 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:33:15.212366 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:33:15.212366 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:33:15.213269 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:33:15.213269 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:33:15.213269 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:33:15.213269 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:33:15.213269 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:15.213269 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:15.213269 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:15.213269 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:15.213269 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:15.213269 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:15.213269 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:15.213269 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:15.213269 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:15.213269 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:15.213269 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:15.213269 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:15.214214 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:15.214214 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:15.214214 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:15.214214 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:15.214214 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:33:15.214214 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:33:15.214214 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:33:15.214214 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:33:15.214214 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:33:15.214214 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:33:15.214214 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 10:33:15.214214 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:33:15.214214 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 143/287] 10.222.0.15 () {56 vars in 1099 bytes} [Sun Sep 11 10:33:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 10:33:15.220157 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be55f0b-79e1-409b-aaf9-1a92bf1b0694 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:15.276081 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:15.361434 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ac01e7-4e60-4e44-80b1-f6c29bd2ebef None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:15.393029 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be55f0b-79e1-409b-aaf9-1a92bf1b0694 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:15.431923 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb044a39-485b-4e38-818c-b5db7523da64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:15.432830 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb044a39-485b-4e38-818c-b5db7523da64 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:15.433294 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb044a39-485b-4e38-818c-b5db7523da64 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:15.433576 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb044a39-485b-4e38-818c-b5db7523da64 None admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:15.434252 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb044a39-485b-4e38-818c-b5db7523da64 None admin] RBAC: Adding query filter params (name=member) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:15.434564 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb044a39-485b-4e38-818c-b5db7523da64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:15.435990 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb044a39-485b-4e38-818c-b5db7523da64 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:15.445971 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 144/288] 10.222.0.15 () {56 vars in 1101 bytes} [Sun Sep 11 10:33:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:15.451956 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb044a39-485b-4e38-818c-b5db7523da64 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:15.474448 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886798c4-085e-4717-8453-4673ad9c6a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:15.475088 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886798c4-085e-4717-8453-4673ad9c6a10 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:15.475366 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886798c4-085e-4717-8453-4673ad9c6a10 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:15.476729 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:15.479732 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886798c4-085e-4717-8453-4673ad9c6a10 None admin] PATH_INFO: `/v3/projects/6d140898a8684238bddc9872a32eb94c` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:15.480963 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886798c4-085e-4717-8453-4673ad9c6a10 None admin] RBAC: Authorizing `identity:get_project(project_id=6d140898a8684238bddc9872a32eb94c)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:15.482519 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886798c4-085e-4717-8453-4673ad9c6a10 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:15.488570 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 145/289] 10.222.0.15 () {56 vars in 1186 bytes} [Sun Sep 11 10:33:15 2022] GET /identity/v3/projects/6d140898a8684238bddc9872a32eb94c => generated 302 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:33:15.508583 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886798c4-085e-4717-8453-4673ad9c6a10 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:15.569048 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb044a39-485b-4e38-818c-b5db7523da64 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:15.609369 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558a9f68-ad25-45dd-a928-ef4de5e06696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:15.610032 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558a9f68-ad25-45dd-a928-ef4de5e06696 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:15.610321 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558a9f68-ad25-45dd-a928-ef4de5e06696 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:15.610647 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558a9f68-ad25-45dd-a928-ef4de5e06696 None admin] PATH_INFO: `/v3/users/5e1ca49806f84e66a9cc647892bcea5c` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:15.616393 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558a9f68-ad25-45dd-a928-ef4de5e06696 None admin] RBAC: Authorizing `identity:get_user(user_id=5e1ca49806f84e66a9cc647892bcea5c)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:15.618480 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558a9f68-ad25-45dd-a928-ef4de5e06696 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:15.625789 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 145/290] 10.222.0.15 () {56 vars in 1177 bytes} [Sun Sep 11 10:33:15 2022] GET /identity/v3/users/5e1ca49806f84e66a9cc647892bcea5c => generated 297 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:33:15.628430 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558a9f68-ad25-45dd-a928-ef4de5e06696 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:15.759662 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886798c4-085e-4717-8453-4673ad9c6a10 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:15.779014 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:15.784455 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:15.815083 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6694715c-f76d-4ce2-b207-6ad06db18ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:15.816083 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6694715c-f76d-4ce2-b207-6ad06db18ee3 None admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:15.816559 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6694715c-f76d-4ce2-b207-6ad06db18ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:15.817005 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6694715c-f76d-4ce2-b207-6ad06db18ee3 None admin] PATH_INFO: `/v3/projects/6d140898a8684238bddc9872a32eb94c/users/f6a0d29dab2243ee8115be5e732de396/roles/d577f4ba2f9c412e9530e50d7f3d6bd7` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:15.821839 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6694715c-f76d-4ce2-b207-6ad06db18ee3 None admin] RBAC: Authorizing `identity:create_grant(project_id=6d140898a8684238bddc9872a32eb94c, user_id=f6a0d29dab2243ee8115be5e732de396, role_id=d577f4ba2f9c412e9530e50d7f3d6bd7)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:15.823497 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6694715c-f76d-4ce2-b207-6ad06db18ee3 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:15.829601 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558a9f68-ad25-45dd-a928-ef4de5e06696 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:15.848838 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 146/291] 10.222.0.15 () {58 vars in 1439 bytes} [Sun Sep 11 10:33:15 2022] PUT /identity/v3/projects/6d140898a8684238bddc9872a32eb94c/users/f6a0d29dab2243ee8115be5e732de396/roles/d577f4ba2f9c412e9530e50d7f3d6bd7 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:33:15.884206 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bde46bb-39e3-4c42-b6a0-dee0e08ce4c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:15.885164 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bde46bb-39e3-4c42-b6a0-dee0e08ce4c2 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:15.885619 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bde46bb-39e3-4c42-b6a0-dee0e08ce4c2 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:15.886062 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bde46bb-39e3-4c42-b6a0-dee0e08ce4c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:15.964164 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bde46bb-39e3-4c42-b6a0-dee0e08ce4c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:15.965984 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bde46bb-39e3-4c42-b6a0-dee0e08ce4c2 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:16.058091 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:16.079524 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 146/292] 10.222.0.15 () {58 vars in 1282 bytes} [Sun Sep 11 10:33:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 10:33:16.088103 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6694715c-f76d-4ce2-b207-6ad06db18ee3 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:16.233818 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6694715c-f76d-4ce2-b207-6ad06db18ee3 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:16.269933 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec8bdd6-7d27-4896-a2e8-8742b88d3ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:16.270874 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec8bdd6-7d27-4896-a2e8-8742b88d3ff1 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:16.271329 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec8bdd6-7d27-4896-a2e8-8742b88d3ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:16.271826 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec8bdd6-7d27-4896-a2e8-8742b88d3ff1 None admin] PATH_INFO: `/v3/projects/6d140898a8684238bddc9872a32eb94c` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:16.273123 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec8bdd6-7d27-4896-a2e8-8742b88d3ff1 None admin] RBAC: Authorizing `identity:get_project(project_id=6d140898a8684238bddc9872a32eb94c)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:16.275271 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec8bdd6-7d27-4896-a2e8-8742b88d3ff1 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:16.279212 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 147/293] 10.222.0.15 () {56 vars in 1186 bytes} [Sun Sep 11 10:33:16 2022] GET /identity/v3/projects/6d140898a8684238bddc9872a32eb94c => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:16.288486 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bde46bb-39e3-4c42-b6a0-dee0e08ce4c2 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:16.393254 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bde46bb-39e3-4c42-b6a0-dee0e08ce4c2 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:16.432181 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfc3fdb-728e-4239-aa4d-e939f03a325a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:16.432788 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc3fdb-728e-4239-aa4d-e939f03a325a None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:16.433070 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc3fdb-728e-4239-aa4d-e939f03a325a None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:16.433341 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc3fdb-728e-4239-aa4d-e939f03a325a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:16.434062 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfc3fdb-728e-4239-aa4d-e939f03a325a None admin] RBAC: Adding query filter params (role.id=f836487a66c74c05838133c9d0614bbf, scope.project.id=6d140898a8684238bddc9872a32eb94c, user.id=5e1ca49806f84e66a9cc647892bcea5c) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:16.434367 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfc3fdb-728e-4239-aa4d-e939f03a325a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:16.435814 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfc3fdb-728e-4239-aa4d-e939f03a325a None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:16.442411 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 147/294] 10.222.0.15 () {56 vars in 1374 bytes} [Sun Sep 11 10:33:16 2022] GET /identity/v3/role_assignments?user.id=5e1ca49806f84e66a9cc647892bcea5c&scope.project.id=6d140898a8684238bddc9872a32eb94c&role.id=f836487a66c74c05838133c9d0614bbf => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:17.043545 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614a9f5f-96d7-4f5c-86b4-5b069cbd14a7 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:17.044638 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614a9f5f-96d7-4f5c-86b4-5b069cbd14a7 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:17.045117 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614a9f5f-96d7-4f5c-86b4-5b069cbd14a7 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:17.046670 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 148/295] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:33:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:33:17.052210 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25fa22-5030-4304-a8c1-390cb1a87a47 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:17.052734 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25fa22-5030-4304-a8c1-390cb1a87a47 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:17.053243 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25fa22-5030-4304-a8c1-390cb1a87a47 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:17.053807 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd81af3c-16ba-48f3-b2d6-abd49a522d5f None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:17.054213 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd81af3c-16ba-48f3-b2d6-abd49a522d5f None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:17.054367 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd81af3c-16ba-48f3-b2d6-abd49a522d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:17.056104 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 149/296] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:33:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 10:33:17.063901 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cc132b-a9e1-4c5a-a83a-2b943b61b0b1 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:17.064187 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cc132b-a9e1-4c5a-a83a-2b943b61b0b1 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:17.064468 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cc132b-a9e1-4c5a-a83a-2b943b61b0b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:17.078365 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-bd81af3c-16ba-48f3-b2d6-abd49a522d5f None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:33:17.080593 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-bd81af3c-16ba-48f3-b2d6-abd49a522d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:33:17.083107 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-c8cc132b-a9e1-4c5a-a83a-2b943b61b0b1 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:33:17.087674 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-c8cc132b-a9e1-4c5a-a83a-2b943b61b0b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:33:17.140091 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:17.147353 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:17.177063 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 150/297] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:33:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 10:33:17.184895 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 148/298] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:33:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:33:17.448027 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cc132b-a9e1-4c5a-a83a-2b943b61b0b1 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:17.481390 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd81af3c-16ba-48f3-b2d6-abd49a522d5f None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:17.564734 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cc132b-a9e1-4c5a-a83a-2b943b61b0b1 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:17.605201 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd81af3c-16ba-48f3-b2d6-abd49a522d5f None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:17.611940 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4242440d-7d0b-4c1b-9bc6-95a6778f7d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:17.612844 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4242440d-7d0b-4c1b-9bc6-95a6778f7d4e None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:17.613325 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4242440d-7d0b-4c1b-9bc6-95a6778f7d4e None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:17.613795 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4242440d-7d0b-4c1b-9bc6-95a6778f7d4e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:17.626436 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4242440d-7d0b-4c1b-9bc6-95a6778f7d4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:33:17.631604 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-4242440d-7d0b-4c1b-9bc6-95a6778f7d4e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 10:33:17.631604 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:17.631604 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:17.631604 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:17.631604 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:17.631604 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:17.631604 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:17.631604 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:17.631604 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:17.631604 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:17.631604 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:17.631604 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:17.631604 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:33:17.631604 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:33:17.631604 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:33:17.631604 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:33:17.632522 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:33:17.632522 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:33:17.632522 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:33:17.632522 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:33:17.632522 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:17.632522 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:17.632522 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:17.632522 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:17.632522 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:17.632522 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:17.632522 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:17.632522 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:17.632522 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:17.632522 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:17.632522 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:17.632522 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:17.633614 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:17.633614 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:17.633614 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:17.633614 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:17.633614 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:33:17.633614 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:33:17.633614 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:33:17.633614 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:33:17.633614 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:33:17.633614 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:33:17.633614 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 10:33:17.633614 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:33:17.636259 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 151/299] 10.222.0.15 () {56 vars in 1099 bytes} [Sun Sep 11 10:33:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:17.642029 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4242440d-7d0b-4c1b-9bc6-95a6778f7d4e None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:17.649899 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19ab432-2efe-4314-a114-6e7b82178bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:17.650601 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19ab432-2efe-4314-a114-6e7b82178bc6 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:17.650889 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19ab432-2efe-4314-a114-6e7b82178bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:17.651165 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19ab432-2efe-4314-a114-6e7b82178bc6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:17.652855 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19ab432-2efe-4314-a114-6e7b82178bc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:17.654367 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19ab432-2efe-4314-a114-6e7b82178bc6 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:17.659497 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 149/300] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:33:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:33:17.663602 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19ab432-2efe-4314-a114-6e7b82178bc6 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:17.692180 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:17.712851 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:17.763291 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4242440d-7d0b-4c1b-9bc6-95a6778f7d4e None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:17.776642 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19ab432-2efe-4314-a114-6e7b82178bc6 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:17.798066 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf0557-2af9-44ce-be09-a7a18d205878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:17.798686 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf0557-2af9-44ce-be09-a7a18d205878 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:17.798967 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf0557-2af9-44ce-be09-a7a18d205878 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:17.799239 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf0557-2af9-44ce-be09-a7a18d205878 None admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:17.799931 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebf0557-2af9-44ce-be09-a7a18d205878 None admin] RBAC: Adding query filter params (name=reader) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:17.800237 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebf0557-2af9-44ce-be09-a7a18d205878 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:17.801842 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebf0557-2af9-44ce-be09-a7a18d205878 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:17.811523 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 152/301] 10.222.0.15 () {56 vars in 1101 bytes} [Sun Sep 11 10:33:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:17.812804 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2ee012-780f-4c64-afdf-610f33f6762d None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:17.813093 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2ee012-780f-4c64-afdf-610f33f6762d None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:17.813365 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2ee012-780f-4c64-afdf-610f33f6762d None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:17.814704 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 153/302] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:33:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:33:17.817138 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665d7f54-e21d-4927-9ea4-fea774f89d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:17.817702 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665d7f54-e21d-4927-9ea4-fea774f89d0f None admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:17.817982 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665d7f54-e21d-4927-9ea4-fea774f89d0f None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:17.818253 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665d7f54-e21d-4927-9ea4-fea774f89d0f None admin] PATH_INFO: `/v3/groups` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:17.818941 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665d7f54-e21d-4927-9ea4-fea774f89d0f None admin] RBAC: Authorizing `identity:create_group()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:17.819521 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2ee012-780f-4c64-afdf-610f33f6762d None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:17.820487 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665d7f54-e21d-4927-9ea4-fea774f89d0f None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:17.847843 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 150/303] 10.222.0.15 () {60 vars in 1134 bytes} [Sun Sep 11 10:33:17 2022] POST /identity/v3/groups => generated 229 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:33:17.849106 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31daf287-4c3a-4c47-8bdf-aef59f1934b6 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:17.849395 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31daf287-4c3a-4c47-8bdf-aef59f1934b6 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:17.849673 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31daf287-4c3a-4c47-8bdf-aef59f1934b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:17.877729 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-31daf287-4c3a-4c47-8bdf-aef59f1934b6 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:33:17.878956 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-31daf287-4c3a-4c47-8bdf-aef59f1934b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:33:17.942306 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:17.949349 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2ee012-780f-4c64-afdf-610f33f6762d None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:17.968634 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 151/304] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:33:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:33:17.994499 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779977a7-f439-4b93-b5e7-5e3afc0043a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:17.995531 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779977a7-f439-4b93-b5e7-5e3afc0043a2 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:17.996026 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779977a7-f439-4b93-b5e7-5e3afc0043a2 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:17.996513 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779977a7-f439-4b93-b5e7-5e3afc0043a2 None admin] PATH_INFO: `/v3/users/26ff5f7a5cf640138fcc95cf6bfc0e0e` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:18.013201 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779977a7-f439-4b93-b5e7-5e3afc0043a2 None admin] RBAC: Authorizing `identity:get_user(user_id=26ff5f7a5cf640138fcc95cf6bfc0e0e)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:18.015093 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779977a7-f439-4b93-b5e7-5e3afc0043a2 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:18.017181 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 154/305] 10.222.0.15 () {56 vars in 1177 bytes} [Sun Sep 11 10:33:17 2022] GET /identity/v3/users/26ff5f7a5cf640138fcc95cf6bfc0e0e => generated 293 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:18.023354 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31daf287-4c3a-4c47-8bdf-aef59f1934b6 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:18.063648 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:18.144110 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31daf287-4c3a-4c47-8bdf-aef59f1934b6 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:18.180677 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8928a48-8bdd-4357-97d9-bd512c30bc22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:18.181632 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8928a48-8bdd-4357-97d9-bd512c30bc22 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:18.182100 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8928a48-8bdd-4357-97d9-bd512c30bc22 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:18.182582 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8928a48-8bdd-4357-97d9-bd512c30bc22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:18.183612 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8928a48-8bdd-4357-97d9-bd512c30bc22 None admin] RBAC: Adding query filter params (role.id=d577f4ba2f9c412e9530e50d7f3d6bd7, user.id=26ff5f7a5cf640138fcc95cf6bfc0e0e, scope.system=all) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:18.184108 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8928a48-8bdd-4357-97d9-bd512c30bc22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:18.185746 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8928a48-8bdd-4357-97d9-bd512c30bc22 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:18.191147 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 152/306] 10.222.0.15 () {56 vars in 1308 bytes} [Sun Sep 11 10:33:18 2022] GET /identity/v3/role_assignments?user.id=26ff5f7a5cf640138fcc95cf6bfc0e0e&scope.system=all&role.id=d577f4ba2f9c412e9530e50d7f3d6bd7 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:18.260579 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779977a7-f439-4b93-b5e7-5e3afc0043a2 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:18.380639 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779977a7-f439-4b93-b5e7-5e3afc0043a2 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:18.416979 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3526cf2c-0f2d-4dca-a81b-c45c1b573d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:18.417699 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3526cf2c-0f2d-4dca-a81b-c45c1b573d94 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:18.417983 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3526cf2c-0f2d-4dca-a81b-c45c1b573d94 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:18.418253 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3526cf2c-0f2d-4dca-a81b-c45c1b573d94 None admin] PATH_INFO: `/v3/roles/member` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:18.428744 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3526cf2c-0f2d-4dca-a81b-c45c1b573d94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:33:18.434415 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-3526cf2c-0f2d-4dca-a81b-c45c1b573d94 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 10:33:18.434415 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:18.434415 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:18.434415 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:18.434415 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:18.434415 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:18.434415 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:18.434415 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:18.434415 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:18.434415 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:18.434415 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:18.434415 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:18.434415 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:33:18.434415 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:33:18.434415 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:33:18.434415 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:33:18.435299 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:33:18.435299 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:33:18.435299 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:33:18.435299 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:33:18.435299 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:18.435299 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:18.435299 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:18.435299 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:18.435299 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:18.435299 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:18.435299 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:18.435299 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:18.435299 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:18.435299 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:18.435299 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:18.435299 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:18.436152 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:18.436152 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:18.436152 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:18.436152 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:18.436152 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:33:18.436152 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:33:18.436152 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:33:18.436152 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:33:18.436152 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:33:18.436152 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:33:18.436152 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 10:33:18.436152 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:33:18.436152 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 155/307] 10.222.0.15 () {56 vars in 1099 bytes} [Sun Sep 11 10:33:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:18.440035 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6fe4f4-c3eb-4c9e-bc37-e972cbceca96 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:18.440323 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6fe4f4-c3eb-4c9e-bc37-e972cbceca96 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:18.440599 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6fe4f4-c3eb-4c9e-bc37-e972cbceca96 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:18.442838 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 153/308] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:33:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:33:18.444018 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3526cf2c-0f2d-4dca-a81b-c45c1b573d94 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:18.447671 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b32e52-1e9e-4134-abbb-402c76bdb696 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:18.447960 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b32e52-1e9e-4134-abbb-402c76bdb696 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:18.448234 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b32e52-1e9e-4134-abbb-402c76bdb696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:18.465246 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:18.467718 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-64b32e52-1e9e-4134-abbb-402c76bdb696 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:33:18.469359 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-64b32e52-1e9e-4134-abbb-402c76bdb696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:33:18.542276 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 154/309] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:33:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 10:33:18.559722 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3526cf2c-0f2d-4dca-a81b-c45c1b573d94 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:18.598085 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c4bb5f-8f80-4fa3-93e1-0c99e4dcea1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:18.598695 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c4bb5f-8f80-4fa3-93e1-0c99e4dcea1b None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:18.598974 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c4bb5f-8f80-4fa3-93e1-0c99e4dcea1b None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:18.599247 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c4bb5f-8f80-4fa3-93e1-0c99e4dcea1b None admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:18.599941 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c4bb5f-8f80-4fa3-93e1-0c99e4dcea1b None admin] RBAC: Adding query filter params (name=member) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:18.600243 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c4bb5f-8f80-4fa3-93e1-0c99e4dcea1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:18.601956 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c4bb5f-8f80-4fa3-93e1-0c99e4dcea1b None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:18.610706 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 156/310] 10.222.0.15 () {56 vars in 1101 bytes} [Sun Sep 11 10:33:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:18.616177 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b32e52-1e9e-4134-abbb-402c76bdb696 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:18.645882 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:18.723206 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b32e52-1e9e-4134-abbb-402c76bdb696 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:18.764058 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3c7943-0dc7-4805-92de-12ddd8132b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:18.764697 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3c7943-0dc7-4805-92de-12ddd8132b33 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:18.764979 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3c7943-0dc7-4805-92de-12ddd8132b33 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:18.765255 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3c7943-0dc7-4805-92de-12ddd8132b33 None admin] PATH_INFO: `/v3/users/60b9fd46b70244c4a289ae23c5e407fb` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:18.766394 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3c7943-0dc7-4805-92de-12ddd8132b33 None admin] RBAC: Authorizing `identity:get_user(user_id=60b9fd46b70244c4a289ae23c5e407fb)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:18.767880 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3c7943-0dc7-4805-92de-12ddd8132b33 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:18.769768 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 155/311] 10.222.0.15 () {56 vars in 1177 bytes} [Sun Sep 11 10:33:18 2022] GET /identity/v3/users/60b9fd46b70244c4a289ae23c5e407fb => generated 293 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:18.775655 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c4bb5f-8f80-4fa3-93e1-0c99e4dcea1b None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:18.853818 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3c7943-0dc7-4805-92de-12ddd8132b33 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:18.859077 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:18.886111 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c4bb5f-8f80-4fa3-93e1-0c99e4dcea1b None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:18.925419 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51474de8-a3b0-4de1-b46b-cf7ca849db1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:18.926508 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51474de8-a3b0-4de1-b46b-cf7ca849db1c None admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:18.926987 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51474de8-a3b0-4de1-b46b-cf7ca849db1c None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:18.927498 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51474de8-a3b0-4de1-b46b-cf7ca849db1c None admin] PATH_INFO: `/v3/system/users/60b9fd46b70244c4a289ae23c5e407fb/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:18.929303 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51474de8-a3b0-4de1-b46b-cf7ca849db1c None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=60b9fd46b70244c4a289ae23c5e407fb, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:18.931170 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51474de8-a3b0-4de1-b46b-cf7ca849db1c None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:18.942572 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 157/312] 10.222.0.15 () {58 vars in 1334 bytes} [Sun Sep 11 10:33:18 2022] PUT /identity/v3/system/users/60b9fd46b70244c4a289ae23c5e407fb/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:33:18.975961 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:18.983986 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3c7943-0dc7-4805-92de-12ddd8132b33 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:19.025368 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e33c582-83d3-44dc-8fa1-9b4a6b4fe7c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:19.026025 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e33c582-83d3-44dc-8fa1-9b4a6b4fe7c5 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:19.026308 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e33c582-83d3-44dc-8fa1-9b4a6b4fe7c5 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:19.026584 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e33c582-83d3-44dc-8fa1-9b4a6b4fe7c5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:19.039126 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e33c582-83d3-44dc-8fa1-9b4a6b4fe7c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:33:19.044629 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-2e33c582-83d3-44dc-8fa1-9b4a6b4fe7c5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 10:33:19.044629 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:19.044629 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:19.044629 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:19.044629 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:19.044629 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:19.044629 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:19.044629 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:19.044629 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:19.044629 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:19.044629 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:19.044629 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:19.044629 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:33:19.044629 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:33:19.044629 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:33:19.044629 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:33:19.045563 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:33:19.045563 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:33:19.045563 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:33:19.045563 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:33:19.045563 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:19.045563 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:19.045563 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:19.045563 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:19.045563 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:19.045563 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:19.045563 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:19.045563 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:19.045563 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:19.045563 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:19.045563 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:19.045563 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:19.046758 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:19.046758 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:19.046758 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:19.046758 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:19.046758 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:33:19.046758 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:33:19.046758 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:33:19.046758 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:33:19.046758 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:33:19.046758 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:33:19.046758 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 10:33:19.046758 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:33:19.046758 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 156/313] 10.222.0.15 () {56 vars in 1099 bytes} [Sun Sep 11 10:33:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:19.053918 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51474de8-a3b0-4de1-b46b-cf7ca849db1c None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:19.175463 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51474de8-a3b0-4de1-b46b-cf7ca849db1c None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:19.212651 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046bd2a7-ba7d-463f-93db-8de018933100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:19.213272 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046bd2a7-ba7d-463f-93db-8de018933100 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:19.213556 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046bd2a7-ba7d-463f-93db-8de018933100 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:19.213831 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046bd2a7-ba7d-463f-93db-8de018933100 None admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:19.214492 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046bd2a7-ba7d-463f-93db-8de018933100 None admin] RBAC: Adding query filter params (name=reader) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:19.214797 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046bd2a7-ba7d-463f-93db-8de018933100 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:19.216265 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046bd2a7-ba7d-463f-93db-8de018933100 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:19.224947 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 158/314] 10.222.0.15 () {56 vars in 1101 bytes} [Sun Sep 11 10:33:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:19.230879 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e33c582-83d3-44dc-8fa1-9b4a6b4fe7c5 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:19.338263 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e33c582-83d3-44dc-8fa1-9b4a6b4fe7c5 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:19.374478 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236ba2c5-4947-447a-b9dc-30dbdf863ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:19.375116 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236ba2c5-4947-447a-b9dc-30dbdf863ff2 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:19.375400 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236ba2c5-4947-447a-b9dc-30dbdf863ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:19.375700 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236ba2c5-4947-447a-b9dc-30dbdf863ff2 None admin] PATH_INFO: `/v3/users/f6a0d29dab2243ee8115be5e732de396` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:19.376830 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236ba2c5-4947-447a-b9dc-30dbdf863ff2 None admin] RBAC: Authorizing `identity:get_user(user_id=f6a0d29dab2243ee8115be5e732de396)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:19.378276 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236ba2c5-4947-447a-b9dc-30dbdf863ff2 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:19.380280 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 157/315] 10.222.0.15 () {56 vars in 1177 bytes} [Sun Sep 11 10:33:19 2022] GET /identity/v3/users/f6a0d29dab2243ee8115be5e732de396 => generated 297 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:19.386003 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046bd2a7-ba7d-463f-93db-8de018933100 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:19.390667 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:19.497612 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046bd2a7-ba7d-463f-93db-8de018933100 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:19.533044 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e5ed41-b000-400c-9d61-8821ce05b646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:19.533734 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e5ed41-b000-400c-9d61-8821ce05b646 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:19.534019 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e5ed41-b000-400c-9d61-8821ce05b646 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:19.534296 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e5ed41-b000-400c-9d61-8821ce05b646 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:19.607718 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e5ed41-b000-400c-9d61-8821ce05b646 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:19.609235 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e5ed41-b000-400c-9d61-8821ce05b646 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:19.611926 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:19.712860 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 159/316] 10.222.0.15 () {58 vars in 1282 bytes} [Sun Sep 11 10:33:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 10:33:19.718513 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236ba2c5-4947-447a-b9dc-30dbdf863ff2 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:19.745799 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:19.823474 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236ba2c5-4947-447a-b9dc-30dbdf863ff2 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:19.859963 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec25c8c-ea17-4f21-8fa2-eb8927981849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:19.860601 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec25c8c-ea17-4f21-8fa2-eb8927981849 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:19.860881 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec25c8c-ea17-4f21-8fa2-eb8927981849 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:19.861151 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec25c8c-ea17-4f21-8fa2-eb8927981849 None admin] PATH_INFO: `/v3/projects/6d140898a8684238bddc9872a32eb94c` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:19.862289 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec25c8c-ea17-4f21-8fa2-eb8927981849 None admin] RBAC: Authorizing `identity:get_project(project_id=6d140898a8684238bddc9872a32eb94c)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:19.863748 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec25c8c-ea17-4f21-8fa2-eb8927981849 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:19.865798 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 158/317] 10.222.0.15 () {56 vars in 1186 bytes} [Sun Sep 11 10:33:19 2022] GET /identity/v3/projects/6d140898a8684238bddc9872a32eb94c => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:19.872229 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e5ed41-b000-400c-9d61-8821ce05b646 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:19.978383 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e5ed41-b000-400c-9d61-8821ce05b646 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:19.980812 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:20.014436 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9217a127-c6df-47ac-9231-bbb380fc51ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:20.015047 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9217a127-c6df-47ac-9231-bbb380fc51ca None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:20.015330 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9217a127-c6df-47ac-9231-bbb380fc51ca None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:20.015635 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9217a127-c6df-47ac-9231-bbb380fc51ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:20.016360 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9217a127-c6df-47ac-9231-bbb380fc51ca None admin] RBAC: Adding query filter params (role.id=d577f4ba2f9c412e9530e50d7f3d6bd7, scope.project.id=6d140898a8684238bddc9872a32eb94c, user.id=f6a0d29dab2243ee8115be5e732de396) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:20.016669 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9217a127-c6df-47ac-9231-bbb380fc51ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:20.018098 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9217a127-c6df-47ac-9231-bbb380fc51ca None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:20.025057 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 160/318] 10.222.0.15 () {56 vars in 1374 bytes} [Sun Sep 11 10:33:19 2022] GET /identity/v3/role_assignments?user.id=f6a0d29dab2243ee8115be5e732de396&scope.project.id=6d140898a8684238bddc9872a32eb94c&role.id=d577f4ba2f9c412e9530e50d7f3d6bd7 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:20.243722 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b2a161-0000-47a2-977d-73e95eef7184 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:20.244689 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b2a161-0000-47a2-977d-73e95eef7184 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:20.245176 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b2a161-0000-47a2-977d-73e95eef7184 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:20.246979 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 159/319] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:33:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:33:20.252749 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7706cf51-8fb0-40b8-a568-7fcf3d4e2296 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:20.253350 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7706cf51-8fb0-40b8-a568-7fcf3d4e2296 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:20.253820 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7706cf51-8fb0-40b8-a568-7fcf3d4e2296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:20.273458 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-7706cf51-8fb0-40b8-a568-7fcf3d4e2296 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:33:20.275015 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-7706cf51-8fb0-40b8-a568-7fcf3d4e2296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:33:20.347732 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 161/320] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:33:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:33:20.467579 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2b77ca-4338-437a-9bef-e0eccce06230 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:20.467857 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2b77ca-4338-437a-9bef-e0eccce06230 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:20.468296 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2b77ca-4338-437a-9bef-e0eccce06230 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:20.469617 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 160/321] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:33:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:33:20.474880 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9592db44-cbf2-4ee1-b744-1fd81f07793b None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:20.475307 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9592db44-cbf2-4ee1-b744-1fd81f07793b None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:20.475520 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9592db44-cbf2-4ee1-b744-1fd81f07793b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:20.494618 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-9592db44-cbf2-4ee1-b744-1fd81f07793b None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:33:20.495739 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-9592db44-cbf2-4ee1-b744-1fd81f07793b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:33:20.562313 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 162/322] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:33:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:33:20.623885 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2b77ca-4338-437a-9bef-e0eccce06230 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:20.720527 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:20.731620 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2b77ca-4338-437a-9bef-e0eccce06230 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:20.769194 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66abc1c-8ae8-4d07-8eb0-7a593113788d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:20.769832 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66abc1c-8ae8-4d07-8eb0-7a593113788d None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:20.770106 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66abc1c-8ae8-4d07-8eb0-7a593113788d None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:20.770373 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66abc1c-8ae8-4d07-8eb0-7a593113788d None admin] PATH_INFO: `/v3/domains/default` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:20.771473 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66abc1c-8ae8-4d07-8eb0-7a593113788d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:20.772911 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66abc1c-8ae8-4d07-8eb0-7a593113788d None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:20.775020 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 161/323] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:33:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:20.780375 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9592db44-cbf2-4ee1-b744-1fd81f07793b None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:20.809969 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:20.830809 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66abc1c-8ae8-4d07-8eb0-7a593113788d None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:20.886618 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9592db44-cbf2-4ee1-b744-1fd81f07793b None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:20.920817 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9530a4-aaf9-4d18-ba4a-13a1f5a323d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:20.921425 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9530a4-aaf9-4d18-ba4a-13a1f5a323d5 None admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:20.921704 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9530a4-aaf9-4d18-ba4a-13a1f5a323d5 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:20.921975 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9530a4-aaf9-4d18-ba4a-13a1f5a323d5 None admin] PATH_INFO: `/v3/groups` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:20.922606 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9530a4-aaf9-4d18-ba4a-13a1f5a323d5 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:20.924068 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9530a4-aaf9-4d18-ba4a-13a1f5a323d5 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:20.937815 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66abc1c-8ae8-4d07-8eb0-7a593113788d None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:20.941604 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 163/324] 10.222.0.15 () {60 vars in 1134 bytes} [Sun Sep 11 10:33:20 2022] POST /identity/v3/groups => generated 226 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 10:33:20.983443 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf14dd0-00b3-4462-9c54-b71a7f957bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:20.984090 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf14dd0-00b3-4462-9c54-b71a7f957bb0 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:20.984367 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf14dd0-00b3-4462-9c54-b71a7f957bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:20.984636 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf14dd0-00b3-4462-9c54-b71a7f957bb0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:20.996535 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf14dd0-00b3-4462-9c54-b71a7f957bb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:33:21.001544 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-9cf14dd0-00b3-4462-9c54-b71a7f957bb0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 10:33:21.001544 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:21.001544 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:21.001544 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:21.001544 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:21.001544 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:21.001544 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:21.001544 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:21.001544 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:21.001544 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:21.001544 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:21.001544 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:21.001544 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:33:21.001544 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:33:21.001544 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:33:21.001544 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:33:21.002565 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:33:21.002565 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:33:21.002565 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:33:21.002565 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:33:21.002565 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:21.002565 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:21.002565 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:21.002565 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:21.002565 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:21.002565 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:21.002565 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:21.002565 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:21.002565 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:21.002565 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:21.002565 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:21.002565 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:21.003379 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:21.003379 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:21.003379 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:21.003379 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:21.003379 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:33:21.003379 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:33:21.003379 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:33:21.003379 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:33:21.003379 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:33:21.003379 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:33:21.003379 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 10:33:21.003379 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:33:21.007777 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 162/325] 10.222.0.15 () {56 vars in 1099 bytes} [Sun Sep 11 10:33:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 10:33:21.009522 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9530a4-aaf9-4d18-ba4a-13a1f5a323d5 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:21.108530 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:21.126854 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9530a4-aaf9-4d18-ba4a-13a1f5a323d5 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:21.165322 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3db697-7937-4f54-a570-c7bb820567ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:21.166003 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3db697-7937-4f54-a570-c7bb820567ae None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:21.166287 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3db697-7937-4f54-a570-c7bb820567ae None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:21.166584 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3db697-7937-4f54-a570-c7bb820567ae None admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:21.167246 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3db697-7937-4f54-a570-c7bb820567ae None admin] RBAC: Adding query filter params (name=reader) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:21.167589 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3db697-7937-4f54-a570-c7bb820567ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:21.168960 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3db697-7937-4f54-a570-c7bb820567ae None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:21.177328 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 164/326] 10.222.0.15 () {56 vars in 1101 bytes} [Sun Sep 11 10:33:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:21.183071 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf14dd0-00b3-4462-9c54-b71a7f957bb0 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:21.210651 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:21.233518 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9252d7-a397-474b-afc2-561291cf5aab None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:21.233813 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9252d7-a397-474b-afc2-561291cf5aab None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:21.234070 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9252d7-a397-474b-afc2-561291cf5aab None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:21.235580 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 165/327] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:33:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 10:33:21.241008 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65b7ad1-13fe-4f7d-82e0-5ec9eeb78e4b None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:21.241477 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65b7ad1-13fe-4f7d-82e0-5ec9eeb78e4b None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:21.241752 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65b7ad1-13fe-4f7d-82e0-5ec9eeb78e4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:21.263111 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-d65b7ad1-13fe-4f7d-82e0-5ec9eeb78e4b None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:33:21.264333 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-d65b7ad1-13fe-4f7d-82e0-5ec9eeb78e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:33:21.318654 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf14dd0-00b3-4462-9c54-b71a7f957bb0 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:21.359526 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 166/328] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:33:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 10:33:21.359690 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05309438-2b38-41c6-a556-efa97d6df91c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:21.360122 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05309438-2b38-41c6-a556-efa97d6df91c None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:21.360410 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05309438-2b38-41c6-a556-efa97d6df91c None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:21.360683 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05309438-2b38-41c6-a556-efa97d6df91c None admin] PATH_INFO: `/v3/users/26ff5f7a5cf640138fcc95cf6bfc0e0e` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:21.361832 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05309438-2b38-41c6-a556-efa97d6df91c None admin] RBAC: Authorizing `identity:get_user(user_id=26ff5f7a5cf640138fcc95cf6bfc0e0e)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:21.363280 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05309438-2b38-41c6-a556-efa97d6df91c None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:21.365031 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 163/329] 10.222.0.15 () {56 vars in 1177 bytes} [Sun Sep 11 10:33:21 2022] GET /identity/v3/users/26ff5f7a5cf640138fcc95cf6bfc0e0e => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:33:21.371431 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65b7ad1-13fe-4f7d-82e0-5ec9eeb78e4b None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:21.465770 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:21.486288 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65b7ad1-13fe-4f7d-82e0-5ec9eeb78e4b None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:21.523283 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8fa3dd-23b7-4642-85d4-748766cc2293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:21.524722 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8fa3dd-23b7-4642-85d4-748766cc2293 None admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:21.525011 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8fa3dd-23b7-4642-85d4-748766cc2293 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:21.525290 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8fa3dd-23b7-4642-85d4-748766cc2293 None admin] PATH_INFO: `/v3/system/users/26ff5f7a5cf640138fcc95cf6bfc0e0e/roles/d577f4ba2f9c412e9530e50d7f3d6bd7` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:21.526730 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8fa3dd-23b7-4642-85d4-748766cc2293 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=26ff5f7a5cf640138fcc95cf6bfc0e0e, role_id=d577f4ba2f9c412e9530e50d7f3d6bd7)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:21.528153 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8fa3dd-23b7-4642-85d4-748766cc2293 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:21.542789 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 167/330] 10.222.0.15 () {58 vars in 1334 bytes} [Sun Sep 11 10:33:21 2022] PUT /identity/v3/system/users/26ff5f7a5cf640138fcc95cf6bfc0e0e/roles/d577f4ba2f9c412e9530e50d7f3d6bd7 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 10:33:21.636672 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05309438-2b38-41c6-a556-efa97d6df91c None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:21.746287 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:21.762592 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05309438-2b38-41c6-a556-efa97d6df91c None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:21.805821 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e6710f-7dca-43f8-9d2f-ec6d33a87e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:21.806834 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e6710f-7dca-43f8-9d2f-ec6d33a87e7b None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:21.807323 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e6710f-7dca-43f8-9d2f-ec6d33a87e7b None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:21.807842 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e6710f-7dca-43f8-9d2f-ec6d33a87e7b None admin] PATH_INFO: `/v3/roles/member` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:21.819632 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e6710f-7dca-43f8-9d2f-ec6d33a87e7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:33:21.830473 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-a7e6710f-7dca-43f8-9d2f-ec6d33a87e7b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 10:33:21.830473 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:21.830473 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:21.830473 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:21.830473 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:21.830473 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:21.830473 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:21.830473 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:21.830473 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:21.830473 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:21.830473 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:21.830473 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:21.830473 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:33:21.830473 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:33:21.830473 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:33:21.830473 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:33:21.831404 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:33:21.831404 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:33:21.831404 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:33:21.831404 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:33:21.831404 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:21.831404 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:21.831404 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:21.831404 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:21.831404 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:21.831404 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:21.831404 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:21.831404 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:21.831404 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:21.831404 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:21.831404 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:21.831404 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:21.832241 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:21.832241 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:21.832241 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:21.832241 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:21.832241 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:33:21.832241 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:33:21.832241 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:33:21.832241 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:33:21.832241 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:33:21.832241 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:33:21.832241 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 10:33:21.832241 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:33:21.834867 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 164/331] 10.222.0.15 () {56 vars in 1099 bytes} [Sun Sep 11 10:33:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 10:33:21.841359 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8fa3dd-23b7-4642-85d4-748766cc2293 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:21.980188 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8fa3dd-23b7-4642-85d4-748766cc2293 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:22.020022 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc15f47-cd6d-4df7-adf7-524a5a3aae97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:22.020648 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc15f47-cd6d-4df7-adf7-524a5a3aae97 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:22.020933 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc15f47-cd6d-4df7-adf7-524a5a3aae97 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:22.021209 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc15f47-cd6d-4df7-adf7-524a5a3aae97 None admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:22.021875 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc15f47-cd6d-4df7-adf7-524a5a3aae97 None admin] RBAC: Adding query filter params (name=member) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:22.022182 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc15f47-cd6d-4df7-adf7-524a5a3aae97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:22.025912 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc15f47-cd6d-4df7-adf7-524a5a3aae97 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:22.037047 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 168/332] 10.222.0.15 () {56 vars in 1101 bytes} [Sun Sep 11 10:33:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:22.042706 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e6710f-7dca-43f8-9d2f-ec6d33a87e7b None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:22.179204 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e6710f-7dca-43f8-9d2f-ec6d33a87e7b None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:22.218970 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cbcefa-f8da-4d9b-ab25-85a21fdf8222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:22.219644 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cbcefa-f8da-4d9b-ab25-85a21fdf8222 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:22.219922 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cbcefa-f8da-4d9b-ab25-85a21fdf8222 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:22.220194 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cbcefa-f8da-4d9b-ab25-85a21fdf8222 None admin] PATH_INFO: `/v3/users/60b9fd46b70244c4a289ae23c5e407fb` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:22.221343 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cbcefa-f8da-4d9b-ab25-85a21fdf8222 None admin] RBAC: Authorizing `identity:get_user(user_id=60b9fd46b70244c4a289ae23c5e407fb)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:22.222802 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cbcefa-f8da-4d9b-ab25-85a21fdf8222 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:22.224418 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 165/333] 10.222.0.15 () {56 vars in 1177 bytes} [Sun Sep 11 10:33:22 2022] GET /identity/v3/users/60b9fd46b70244c4a289ae23c5e407fb => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:22.230811 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc15f47-cd6d-4df7-adf7-524a5a3aae97 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:22.245562 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:22.375349 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc15f47-cd6d-4df7-adf7-524a5a3aae97 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:22.423948 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3e3af6-bd18-4aaa-b226-ccc5630c39a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:22.424565 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e3af6-bd18-4aaa-b226-ccc5630c39a9 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:22.424846 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e3af6-bd18-4aaa-b226-ccc5630c39a9 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:22.425117 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e3af6-bd18-4aaa-b226-ccc5630c39a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:22.425843 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3e3af6-bd18-4aaa-b226-ccc5630c39a9 None admin] RBAC: Adding query filter params (role.id=f836487a66c74c05838133c9d0614bbf, user.id=60b9fd46b70244c4a289ae23c5e407fb, scope.system=all) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:22.426149 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3e3af6-bd18-4aaa-b226-ccc5630c39a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:22.427624 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3e3af6-bd18-4aaa-b226-ccc5630c39a9 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:22.443284 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 169/334] 10.222.0.15 () {56 vars in 1308 bytes} [Sun Sep 11 10:33:22 2022] GET /identity/v3/role_assignments?user.id=60b9fd46b70244c4a289ae23c5e407fb&scope.system=all&role.id=f836487a66c74c05838133c9d0614bbf => generated 510 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:23.534887 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d29fd81-6277-4cf2-b928-7a6fdd8d8106 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:23.535391 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d29fd81-6277-4cf2-b928-7a6fdd8d8106 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:23.535525 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d29fd81-6277-4cf2-b928-7a6fdd8d8106 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:23.537059 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 166/335] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:33:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:33:23.542271 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bbbd36-f7b1-4ba9-b689-d683f5483e8a None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:23.543117 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bbbd36-f7b1-4ba9-b689-d683f5483e8a None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:23.543592 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bbbd36-f7b1-4ba9-b689-d683f5483e8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:23.566737 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-b5bbbd36-f7b1-4ba9-b689-d683f5483e8a None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:33:23.568199 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-b5bbbd36-f7b1-4ba9-b689-d683f5483e8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:33:23.646121 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:23.657326 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 170/336] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:33:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:33:24.103278 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d29fd81-6277-4cf2-b928-7a6fdd8d8106 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:24.124828 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:24.237968 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d29fd81-6277-4cf2-b928-7a6fdd8d8106 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:24.288741 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e74998-1655-4fa6-ac55-eec070cad7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:24.290112 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e74998-1655-4fa6-ac55-eec070cad7b7 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:24.290671 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e74998-1655-4fa6-ac55-eec070cad7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:24.291186 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e74998-1655-4fa6-ac55-eec070cad7b7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:24.305550 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e74998-1655-4fa6-ac55-eec070cad7b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:33:24.310856 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-f2e74998-1655-4fa6-ac55-eec070cad7b7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 10:33:24.310856 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:24.310856 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:24.310856 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:24.310856 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:24.310856 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:24.310856 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:24.310856 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:24.310856 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:24.310856 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:24.310856 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:24.310856 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:24.310856 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:33:24.310856 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:33:24.310856 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:33:24.310856 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:33:24.311756 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:33:24.311756 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:33:24.311756 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:33:24.311756 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:33:24.311756 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:24.311756 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:24.311756 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:24.311756 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:24.311756 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:24.311756 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:24.311756 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:24.311756 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:24.311756 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:24.311756 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:24.311756 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:24.311756 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:24.312538 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:24.312538 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:24.312538 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:24.312538 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:24.312538 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:33:24.312538 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:33:24.312538 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:33:24.312538 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:33:24.312538 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:33:24.312538 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:33:24.312538 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 10:33:24.312538 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:33:24.312538 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 167/337] 10.222.0.15 () {56 vars in 1099 bytes} [Sun Sep 11 10:33:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:24.319254 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bbbd36-f7b1-4ba9-b689-d683f5483e8a None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:24.412323 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87a8b44-2d62-4132-bdfc-880e150a3f16 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:24.412926 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87a8b44-2d62-4132-bdfc-880e150a3f16 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:24.413401 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87a8b44-2d62-4132-bdfc-880e150a3f16 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:24.419958 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 168/338] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:33:24 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 10:33:24.425274 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5589c6-dc8e-4296-97fe-47ebd05e4492 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:24.425786 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5589c6-dc8e-4296-97fe-47ebd05e4492 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:24.426251 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5589c6-dc8e-4296-97fe-47ebd05e4492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:24.454710 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bbbd36-f7b1-4ba9-b689-d683f5483e8a None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:24.465075 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-5e5589c6-dc8e-4296-97fe-47ebd05e4492 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:33:24.471113 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-5e5589c6-dc8e-4296-97fe-47ebd05e4492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:33:24.519515 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd87600-8055-4c9d-ad0c-3b8679114252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:24.520128 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd87600-8055-4c9d-ad0c-3b8679114252 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:24.520405 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd87600-8055-4c9d-ad0c-3b8679114252 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:24.520674 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd87600-8055-4c9d-ad0c-3b8679114252 None admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:24.521331 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd87600-8055-4c9d-ad0c-3b8679114252 None admin] RBAC: Adding query filter params (name=member) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:24.521626 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd87600-8055-4c9d-ad0c-3b8679114252 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:24.522991 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd87600-8055-4c9d-ad0c-3b8679114252 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:24.545885 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 171/339] 10.222.0.15 () {56 vars in 1101 bytes} [Sun Sep 11 10:33:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:24.552182 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd87600-8055-4c9d-ad0c-3b8679114252 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:24.611568 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 169/340] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:33:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:33:24.643381 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:24.699834 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd87600-8055-4c9d-ad0c-3b8679114252 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:24.740069 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12280144-1b4d-48cb-bd23-969222a56911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:24.740758 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12280144-1b4d-48cb-bd23-969222a56911 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:24.741043 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12280144-1b4d-48cb-bd23-969222a56911 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:24.741315 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12280144-1b4d-48cb-bd23-969222a56911 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:24.825696 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12280144-1b4d-48cb-bd23-969222a56911 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:24.827153 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12280144-1b4d-48cb-bd23-969222a56911 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:24.870195 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3631b2-4a2c-44c4-b6b9-a85e8f2eb751 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:24.870494 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3631b2-4a2c-44c4-b6b9-a85e8f2eb751 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:24.870744 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3631b2-4a2c-44c4-b6b9-a85e8f2eb751 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:24.872362 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 170/341] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:33:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:33:24.885072 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba9cd9-8f13-4c4d-8f1e-6aa4aea51726 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:24.885493 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba9cd9-8f13-4c4d-8f1e-6aa4aea51726 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:24.885866 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba9cd9-8f13-4c4d-8f1e-6aa4aea51726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:24.907828 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:24.911810 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:24.919634 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-25ba9cd9-8f13-4c4d-8f1e-6aa4aea51726 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:33:24.920639 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-25ba9cd9-8f13-4c4d-8f1e-6aa4aea51726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:33:24.996134 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 172/342] 10.222.0.15 () {58 vars in 1282 bytes} [Sun Sep 11 10:33:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 10:33:25.001583 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12280144-1b4d-48cb-bd23-969222a56911 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:25.052911 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 171/343] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:33:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:33:25.054154 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51adcae9-8709-44e2-bc02-28906780026a None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:25.054438 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51adcae9-8709-44e2-bc02-28906780026a None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:25.055272 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51adcae9-8709-44e2-bc02-28906780026a None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:25.060073 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 172/344] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:33:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:33:25.067995 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317fcf01-d9ec-46d4-ba93-dfde39408727 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:25.068273 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317fcf01-d9ec-46d4-ba93-dfde39408727 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:25.068542 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317fcf01-d9ec-46d4-ba93-dfde39408727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:25.103141 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-317fcf01-d9ec-46d4-ba93-dfde39408727 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:33:25.105805 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-317fcf01-d9ec-46d4-ba93-dfde39408727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:33:25.173642 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12280144-1b4d-48cb-bd23-969222a56911 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:25.216005 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 173/345] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:33:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:33:25.216863 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317fcf01-d9ec-46d4-ba93-dfde39408727 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:25.243011 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66988e96-72c1-40e1-905b-88a4722e847a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:25.244024 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66988e96-72c1-40e1-905b-88a4722e847a None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:25.244543 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66988e96-72c1-40e1-905b-88a4722e847a None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:25.245018 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66988e96-72c1-40e1-905b-88a4722e847a None admin] PATH_INFO: `/v3/projects/1b12f8d03f4f4f4c8b5f37b804de18c5` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:25.252190 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66988e96-72c1-40e1-905b-88a4722e847a None admin] RBAC: Authorizing `identity:get_project(project_id=1b12f8d03f4f4f4c8b5f37b804de18c5)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:25.254054 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66988e96-72c1-40e1-905b-88a4722e847a None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:25.255929 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 173/346] 10.222.0.15 () {56 vars in 1186 bytes} [Sun Sep 11 10:33:25 2022] GET /identity/v3/projects/1b12f8d03f4f4f4c8b5f37b804de18c5 => generated 298 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:33:25.262210 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66988e96-72c1-40e1-905b-88a4722e847a None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:25.313966 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:25.328828 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:25.387787 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317fcf01-d9ec-46d4-ba93-dfde39408727 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:25.443886 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8c886f-0c84-4e81-8b50-46cf22c423fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:25.446001 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c886f-0c84-4e81-8b50-46cf22c423fd None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:25.446650 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c886f-0c84-4e81-8b50-46cf22c423fd None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:25.447130 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c886f-0c84-4e81-8b50-46cf22c423fd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:25.461446 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8c886f-0c84-4e81-8b50-46cf22c423fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:33:25.467297 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-5b8c886f-0c84-4e81-8b50-46cf22c423fd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 10:33:25.467297 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:25.467297 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:25.467297 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:25.467297 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:25.467297 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:25.467297 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:25.467297 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:25.467297 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:25.467297 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:25.467297 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:25.467297 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:25.467297 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:33:25.467297 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:33:25.467297 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:33:25.467297 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:33:25.468267 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:33:25.468267 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:33:25.468267 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:33:25.468267 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:33:25.468267 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:25.468267 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:25.468267 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:25.468267 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:25.468267 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:25.468267 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:25.468267 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:25.468267 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:25.468267 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:25.468267 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:25.468267 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:25.468267 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:25.469109 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:25.469109 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:25.469109 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:25.469109 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:25.469109 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:33:25.469109 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:33:25.469109 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:33:25.469109 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:33:25.469109 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:33:25.469109 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:33:25.469109 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 10:33:25.469109 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:33:25.474248 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 174/347] 10.222.0.15 () {56 vars in 1114 bytes} [Sun Sep 11 10:33:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:25.475637 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8c886f-0c84-4e81-8b50-46cf22c423fd None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:25.534424 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66988e96-72c1-40e1-905b-88a4722e847a None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:25.609541 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dffafe-3d84-4a8d-b937-ec9323c9c4a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:25.610199 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dffafe-3d84-4a8d-b937-ec9323c9c4a0 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:25.610459 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dffafe-3d84-4a8d-b937-ec9323c9c4a0 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:25.610723 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dffafe-3d84-4a8d-b937-ec9323c9c4a0 None admin] PATH_INFO: `/v3/groups/30c24407ddbb4d64af64c09b373e5a25` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:25.624128 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dffafe-3d84-4a8d-b937-ec9323c9c4a0 None admin] RBAC: Authorizing `identity:get_group(group_id=30c24407ddbb4d64af64c09b373e5a25)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:25.625605 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dffafe-3d84-4a8d-b937-ec9323c9c4a0 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:25.627291 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 174/348] 10.222.0.15 () {56 vars in 1180 bytes} [Sun Sep 11 10:33:25 2022] GET /identity/v3/groups/30c24407ddbb4d64af64c09b373e5a25 => generated 226 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:33:25.631614 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dffafe-3d84-4a8d-b937-ec9323c9c4a0 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:25.643316 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8c886f-0c84-4e81-8b50-46cf22c423fd None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:25.679143 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:25.697505 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3c6ce6-a053-4435-969a-3356d1c325f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:25.698147 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3c6ce6-a053-4435-969a-3356d1c325f0 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:25.698416 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3c6ce6-a053-4435-969a-3356d1c325f0 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:25.698687 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3c6ce6-a053-4435-969a-3356d1c325f0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:25.711523 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3c6ce6-a053-4435-969a-3356d1c325f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:33:25.717003 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-4e3c6ce6-a053-4435-969a-3356d1c325f0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 10:33:25.717003 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:25.717003 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:25.717003 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:25.717003 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:25.717003 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:25.717003 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:25.717003 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:25.717003 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:25.717003 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:25.717003 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:25.717003 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:25.717003 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:33:25.717003 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:33:25.717003 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:33:25.717003 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:33:25.717948 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:33:25.717948 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:33:25.717948 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:33:25.717948 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:33:25.717948 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:25.717948 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:25.717948 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:25.717948 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:25.717948 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:25.717948 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:25.717948 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:25.717948 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:25.717948 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:25.717948 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:25.717948 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:25.717948 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:25.718756 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:25.718756 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:25.718756 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:25.718756 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:25.718756 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:33:25.718756 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:33:25.718756 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:33:25.718756 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:33:25.718756 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:33:25.718756 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:33:25.718756 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 10:33:25.718756 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:33:25.719379 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 175/349] 10.222.0.15 () {56 vars in 1099 bytes} [Sun Sep 11 10:33:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:25.719940 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3c6ce6-a053-4435-969a-3356d1c325f0 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:25.764448 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:25.785740 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dffafe-3d84-4a8d-b937-ec9323c9c4a0 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:25.837835 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0561492-c7d5-45cd-8310-bcdece7821fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:25.838450 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0561492-c7d5-45cd-8310-bcdece7821fd None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:25.838731 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0561492-c7d5-45cd-8310-bcdece7821fd None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:25.839003 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0561492-c7d5-45cd-8310-bcdece7821fd None admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:25.839743 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0561492-c7d5-45cd-8310-bcdece7821fd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:25.840046 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0561492-c7d5-45cd-8310-bcdece7821fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:25.841434 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0561492-c7d5-45cd-8310-bcdece7821fd None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:25.858906 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 175/350] 10.222.0.15 () {56 vars in 1111 bytes} [Sun Sep 11 10:33:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:25.858906 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0561492-c7d5-45cd-8310-bcdece7821fd None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:25.869728 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3c6ce6-a053-4435-969a-3356d1c325f0 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:25.909842 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801fff99-56b0-4473-b76f-e8560f2460af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:25.910507 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801fff99-56b0-4473-b76f-e8560f2460af None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:25.910839 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801fff99-56b0-4473-b76f-e8560f2460af None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:25.911109 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801fff99-56b0-4473-b76f-e8560f2460af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:25.911915 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801fff99-56b0-4473-b76f-e8560f2460af None admin] RBAC: Adding query filter params (group.id=30c24407ddbb4d64af64c09b373e5a25, role.id=f836487a66c74c05838133c9d0614bbf, scope.project.id=1b12f8d03f4f4f4c8b5f37b804de18c5) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:25.912223 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801fff99-56b0-4473-b76f-e8560f2460af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:25.913658 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801fff99-56b0-4473-b76f-e8560f2460af None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:25.920457 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 176/351] 10.222.0.15 () {56 vars in 1376 bytes} [Sun Sep 11 10:33:25 2022] GET /identity/v3/role_assignments?group.id=30c24407ddbb4d64af64c09b373e5a25&scope.project.id=1b12f8d03f4f4f4c8b5f37b804de18c5&role.id=f836487a66c74c05838133c9d0614bbf => generated 260 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:25.921311 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801fff99-56b0-4473-b76f-e8560f2460af None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:26.015959 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0561492-c7d5-45cd-8310-bcdece7821fd None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:26.026726 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:26.072233 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9292b31-5fbe-46c1-b577-2c8fe1e44655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:26.072864 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9292b31-5fbe-46c1-b577-2c8fe1e44655 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:26.073147 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9292b31-5fbe-46c1-b577-2c8fe1e44655 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:26.073423 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9292b31-5fbe-46c1-b577-2c8fe1e44655 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:26.091396 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9292b31-5fbe-46c1-b577-2c8fe1e44655 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:33:26.098288 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-e9292b31-5fbe-46c1-b577-2c8fe1e44655 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 10:33:26.098288 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:26.098288 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:26.098288 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:26.098288 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:26.098288 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:26.098288 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:26.098288 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:26.098288 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:26.098288 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:26.098288 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:26.098288 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:26.098288 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:33:26.098288 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:33:26.098288 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:33:26.098288 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:33:26.099271 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:33:26.099271 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:33:26.099271 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:33:26.099271 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:33:26.099271 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:26.099271 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:26.099271 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:26.099271 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:26.099271 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:26.099271 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:26.099271 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:26.099271 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:26.099271 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:26.099271 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:26.099271 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:26.099271 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:26.100141 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:26.100141 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:26.100141 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:26.100141 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:26.100141 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:33:26.100141 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:33:26.100141 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:33:26.100141 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:33:26.100141 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:33:26.100141 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:33:26.100141 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 10:33:26.100141 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:33:26.100777 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 176/352] 10.222.0.15 () {56 vars in 1099 bytes} [Sun Sep 11 10:33:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:26.101380 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9292b31-5fbe-46c1-b577-2c8fe1e44655 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:26.124644 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:26.165567 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801fff99-56b0-4473-b76f-e8560f2460af None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:26.220480 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033a949a-0e4e-4d60-b3b8-c4310b850426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:26.221499 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033a949a-0e4e-4d60-b3b8-c4310b850426 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:26.222001 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033a949a-0e4e-4d60-b3b8-c4310b850426 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:26.222495 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033a949a-0e4e-4d60-b3b8-c4310b850426 None admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:26.223392 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033a949a-0e4e-4d60-b3b8-c4310b850426 None admin] RBAC: Adding query filter params (name=member) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:26.223951 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033a949a-0e4e-4d60-b3b8-c4310b850426 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:26.227196 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033a949a-0e4e-4d60-b3b8-c4310b850426 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:26.239552 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 177/353] 10.222.0.15 () {56 vars in 1101 bytes} [Sun Sep 11 10:33:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:26.241304 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033a949a-0e4e-4d60-b3b8-c4310b850426 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:26.294368 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9292b31-5fbe-46c1-b577-2c8fe1e44655 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:26.346517 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0b2af0-2734-444d-b11c-9171da2a08bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:26.346517 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b2af0-2734-444d-b11c-9171da2a08bc None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:26.346784 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b2af0-2734-444d-b11c-9171da2a08bc None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:26.346861 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b2af0-2734-444d-b11c-9171da2a08bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:26.389578 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033a949a-0e4e-4d60-b3b8-c4310b850426 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:26.408826 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:26.437116 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0b2af0-2734-444d-b11c-9171da2a08bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:26.439262 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0b2af0-2734-444d-b11c-9171da2a08bc None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:26.443011 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c546e72f-0848-4569-a202-8a72518eaa77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:26.443532 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c546e72f-0848-4569-a202-8a72518eaa77 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:26.443818 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c546e72f-0848-4569-a202-8a72518eaa77 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:26.444084 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c546e72f-0848-4569-a202-8a72518eaa77 None admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:26.444750 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c546e72f-0848-4569-a202-8a72518eaa77 None admin] RBAC: Adding query filter params (name=reader) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:26.445056 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c546e72f-0848-4569-a202-8a72518eaa77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:26.447158 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c546e72f-0848-4569-a202-8a72518eaa77 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:26.456950 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 178/354] 10.222.0.15 () {56 vars in 1101 bytes} [Sun Sep 11 10:33:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:26.457783 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c546e72f-0848-4569-a202-8a72518eaa77 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:26.458180 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:26.575967 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 177/355] 10.222.0.15 () {58 vars in 1282 bytes} [Sun Sep 11 10:33:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 10:33:26.577573 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487e840f-83f7-4671-8ea0-208a8fcf0ee0 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:26.577693 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487e840f-83f7-4671-8ea0-208a8fcf0ee0 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:26.577876 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487e840f-83f7-4671-8ea0-208a8fcf0ee0 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:26.579187 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 178/356] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:33:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:33:26.582194 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c255fd17-9eac-4c51-a493-fe7c85ab771f None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:26.582194 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c255fd17-9eac-4c51-a493-fe7c85ab771f None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:26.582194 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c255fd17-9eac-4c51-a493-fe7c85ab771f None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:26.582194 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 179/357] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:33:26 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:33:26.582659 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c255fd17-9eac-4c51-a493-fe7c85ab771f None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:26.595985 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c546e72f-0848-4569-a202-8a72518eaa77 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:26.643886 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a9df0c-d163-41c9-a3a0-a2520a31280c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:26.644508 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a9df0c-d163-41c9-a3a0-a2520a31280c None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:26.644747 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a9df0c-d163-41c9-a3a0-a2520a31280c None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:26.644944 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a9df0c-d163-41c9-a3a0-a2520a31280c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:26.703673 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:26.728373 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c255fd17-9eac-4c51-a493-fe7c85ab771f None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:26.741224 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:26.747487 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a9df0c-d163-41c9-a3a0-a2520a31280c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:26.750292 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a9df0c-d163-41c9-a3a0-a2520a31280c None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:26.767799 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7b5580-e903-461e-bea8-ded1bdab1b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:26.768445 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7b5580-e903-461e-bea8-ded1bdab1b8b None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:26.768735 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7b5580-e903-461e-bea8-ded1bdab1b8b None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:26.769011 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7b5580-e903-461e-bea8-ded1bdab1b8b None admin] PATH_INFO: `/v3/users/26ff5f7a5cf640138fcc95cf6bfc0e0e` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:26.770941 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7b5580-e903-461e-bea8-ded1bdab1b8b None admin] RBAC: Authorizing `identity:get_user(user_id=26ff5f7a5cf640138fcc95cf6bfc0e0e)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:26.772430 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7b5580-e903-461e-bea8-ded1bdab1b8b None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:26.774574 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 180/358] 10.222.0.15 () {56 vars in 1177 bytes} [Sun Sep 11 10:33:26 2022] GET /identity/v3/users/26ff5f7a5cf640138fcc95cf6bfc0e0e => generated 293 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:26.775351 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7b5580-e903-461e-bea8-ded1bdab1b8b None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:26.878065 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 179/359] 10.222.0.15 () {58 vars in 1282 bytes} [Sun Sep 11 10:33:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 10:33:26.879327 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a828bd0b-066c-4439-8fac-58513c038970 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:26.879681 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a828bd0b-066c-4439-8fac-58513c038970 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:26.879945 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a828bd0b-066c-4439-8fac-58513c038970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:26.903601 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-a828bd0b-066c-4439-8fac-58513c038970 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:33:26.904725 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-a828bd0b-066c-4439-8fac-58513c038970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:33:26.911342 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7b5580-e903-461e-bea8-ded1bdab1b8b None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:26.949643 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf6de09-3cfd-4d78-91f9-d6d4705e8722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:26.950310 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf6de09-3cfd-4d78-91f9-d6d4705e8722 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:26.950593 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf6de09-3cfd-4d78-91f9-d6d4705e8722 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:26.952001 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf6de09-3cfd-4d78-91f9-d6d4705e8722 None admin] PATH_INFO: `/v3/projects/1b12f8d03f4f4f4c8b5f37b804de18c5` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:26.954931 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf6de09-3cfd-4d78-91f9-d6d4705e8722 None admin] RBAC: Authorizing `identity:get_project(project_id=1b12f8d03f4f4f4c8b5f37b804de18c5)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:26.954931 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf6de09-3cfd-4d78-91f9-d6d4705e8722 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:26.956413 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 181/360] 10.222.0.15 () {56 vars in 1186 bytes} [Sun Sep 11 10:33:26 2022] GET /identity/v3/projects/1b12f8d03f4f4f4c8b5f37b804de18c5 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:26.957224 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f2df69-80d1-4bca-b7b3-75b0027f1c89 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:26.957508 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f2df69-80d1-4bca-b7b3-75b0027f1c89 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:26.958081 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f2df69-80d1-4bca-b7b3-75b0027f1c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:26.979949 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-c7f2df69-80d1-4bca-b7b3-75b0027f1c89 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:33:26.981155 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-c7f2df69-80d1-4bca-b7b3-75b0027f1c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:33:26.995470 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 180/361] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:33:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:33:26.996345 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a828bd0b-066c-4439-8fac-58513c038970 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:27.007039 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:27.028586 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:27.056240 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 182/362] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:33:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:33:27.056905 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f2df69-80d1-4bca-b7b3-75b0027f1c89 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:27.149888 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a828bd0b-066c-4439-8fac-58513c038970 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:27.197842 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3893f885-2a94-4ae9-a5f1-48a600cf6723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:27.198373 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3893f885-2a94-4ae9-a5f1-48a600cf6723 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:27.198656 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3893f885-2a94-4ae9-a5f1-48a600cf6723 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:27.198928 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3893f885-2a94-4ae9-a5f1-48a600cf6723 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:27.199680 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3893f885-2a94-4ae9-a5f1-48a600cf6723 None admin] RBAC: Adding query filter params (role.id=d577f4ba2f9c412e9530e50d7f3d6bd7, user.id=26ff5f7a5cf640138fcc95cf6bfc0e0e, scope.system=all) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:27.200266 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3893f885-2a94-4ae9-a5f1-48a600cf6723 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:27.201722 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3893f885-2a94-4ae9-a5f1-48a600cf6723 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:27.227323 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 181/363] 10.222.0.15 () {56 vars in 1308 bytes} [Sun Sep 11 10:33:26 2022] GET /identity/v3/role_assignments?user.id=26ff5f7a5cf640138fcc95cf6bfc0e0e&scope.system=all&role.id=d577f4ba2f9c412e9530e50d7f3d6bd7 => generated 510 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:27.228258 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3893f885-2a94-4ae9-a5f1-48a600cf6723 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:27.274174 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f2df69-80d1-4bca-b7b3-75b0027f1c89 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:27.365095 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2a0e4a-e66d-43de-b896-f2bf0a959e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:27.365753 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2a0e4a-e66d-43de-b896-f2bf0a959e17 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:27.366045 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2a0e4a-e66d-43de-b896-f2bf0a959e17 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:27.366318 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2a0e4a-e66d-43de-b896-f2bf0a959e17 None admin] PATH_INFO: `/v3/projects/6d140898a8684238bddc9872a32eb94c` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:27.368350 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2a0e4a-e66d-43de-b896-f2bf0a959e17 None admin] RBAC: Authorizing `identity:get_project(project_id=6d140898a8684238bddc9872a32eb94c)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:27.369847 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2a0e4a-e66d-43de-b896-f2bf0a959e17 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:27.371890 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 183/364] 10.222.0.15 () {56 vars in 1186 bytes} [Sun Sep 11 10:33:27 2022] GET /identity/v3/projects/6d140898a8684238bddc9872a32eb94c => generated 302 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:27.377887 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2a0e4a-e66d-43de-b896-f2bf0a959e17 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:27.379765 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:27.446214 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3893f885-2a94-4ae9-a5f1-48a600cf6723 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:27.490149 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:27.502921 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dca2db1-c63f-4ddb-a829-1221576fd265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:27.504092 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca2db1-c63f-4ddb-a829-1221576fd265 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:27.504380 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca2db1-c63f-4ddb-a829-1221576fd265 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:27.504713 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca2db1-c63f-4ddb-a829-1221576fd265 None admin] PATH_INFO: `/v3/groups/30c24407ddbb4d64af64c09b373e5a25` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:27.505927 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dca2db1-c63f-4ddb-a829-1221576fd265 None admin] RBAC: Authorizing `identity:get_group(group_id=30c24407ddbb4d64af64c09b373e5a25)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:27.508346 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dca2db1-c63f-4ddb-a829-1221576fd265 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:27.510900 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 182/365] 10.222.0.15 () {56 vars in 1180 bytes} [Sun Sep 11 10:33:27 2022] GET /identity/v3/groups/30c24407ddbb4d64af64c09b373e5a25 => generated 226 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:27.511906 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dca2db1-c63f-4ddb-a829-1221576fd265 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:27.569673 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2a0e4a-e66d-43de-b896-f2bf0a959e17 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:27.612352 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ec7a16-f24f-4674-b381-f5aca4ad7587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:27.613006 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ec7a16-f24f-4674-b381-f5aca4ad7587 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:27.613267 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ec7a16-f24f-4674-b381-f5aca4ad7587 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:27.613541 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ec7a16-f24f-4674-b381-f5aca4ad7587 None admin] PATH_INFO: `/v3/groups/30c24407ddbb4d64af64c09b373e5a25` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:27.614780 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ec7a16-f24f-4674-b381-f5aca4ad7587 None admin] RBAC: Authorizing `identity:get_group(group_id=30c24407ddbb4d64af64c09b373e5a25)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:27.617249 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ec7a16-f24f-4674-b381-f5aca4ad7587 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:27.618893 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 184/366] 10.222.0.15 () {56 vars in 1180 bytes} [Sun Sep 11 10:33:27 2022] GET /identity/v3/groups/30c24407ddbb4d64af64c09b373e5a25 => generated 226 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:27.620448 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ec7a16-f24f-4674-b381-f5aca4ad7587 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:27.646330 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dca2db1-c63f-4ddb-a829-1221576fd265 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:27.688235 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:27.694791 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f69999d-87f5-49f3-bee9-dc5c8d5aaf73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:27.696020 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f69999d-87f5-49f3-bee9-dc5c8d5aaf73 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:27.696336 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f69999d-87f5-49f3-bee9-dc5c8d5aaf73 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:27.696609 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f69999d-87f5-49f3-bee9-dc5c8d5aaf73 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:27.715658 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f69999d-87f5-49f3-bee9-dc5c8d5aaf73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:33:27.721359 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-3f69999d-87f5-49f3-bee9-dc5c8d5aaf73 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 10:33:27.721359 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:27.721359 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:27.721359 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:27.721359 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:27.721359 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:27.721359 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:27.721359 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:27.721359 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:27.721359 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:27.721359 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:27.721359 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:27.721359 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:33:27.721359 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:33:27.721359 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:33:27.721359 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:33:27.722240 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:33:27.722240 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:33:27.722240 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:33:27.722240 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:33:27.722240 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:27.722240 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:27.722240 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:27.722240 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:27.722240 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:27.722240 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:27.722240 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:27.722240 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:27.722240 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:27.722240 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:27.722240 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:27.722240 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:27.723044 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:27.723044 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:27.723044 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:27.723044 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:27.723044 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:33:27.723044 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:33:27.723044 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:33:27.723044 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:33:27.723044 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:33:27.723044 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:33:27.723044 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 10:33:27.723044 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:33:27.725915 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 183/367] 10.222.0.15 () {56 vars in 1114 bytes} [Sun Sep 11 10:33:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:27.728079 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f69999d-87f5-49f3-bee9-dc5c8d5aaf73 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:27.747090 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ec7a16-f24f-4674-b381-f5aca4ad7587 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:27.766015 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:27.794295 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d66af5e-0526-4fa2-9bdc-3e568178de63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:27.795248 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d66af5e-0526-4fa2-9bdc-3e568178de63 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:27.795563 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d66af5e-0526-4fa2-9bdc-3e568178de63 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:27.796430 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d66af5e-0526-4fa2-9bdc-3e568178de63 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:27.811468 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d66af5e-0526-4fa2-9bdc-3e568178de63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:33:27.817662 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-2d66af5e-0526-4fa2-9bdc-3e568178de63 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 10:33:27.817662 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:27.817662 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:27.817662 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:27.817662 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:27.817662 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:27.817662 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:27.817662 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:27.817662 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:27.817662 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:27.817662 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:27.817662 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:27.817662 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:33:27.817662 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:33:27.817662 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:33:27.817662 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:33:27.818570 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:33:27.818570 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:33:27.818570 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:33:27.818570 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:33:27.818570 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:27.818570 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:27.818570 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:27.818570 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:27.818570 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:27.818570 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:27.818570 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:27.818570 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:27.818570 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:27.818570 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:27.818570 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:27.818570 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:27.819382 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:27.819382 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:27.819382 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:27.819382 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:27.819382 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:33:27.819382 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:33:27.819382 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:33:27.819382 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:33:27.819382 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:33:27.819382 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:33:27.819382 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 10:33:27.819382 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:33:27.819382 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 185/368] 10.222.0.15 () {56 vars in 1096 bytes} [Sun Sep 11 10:33:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:27.820106 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d66af5e-0526-4fa2-9bdc-3e568178de63 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:27.868396 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f69999d-87f5-49f3-bee9-dc5c8d5aaf73 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:27.912629 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e947af6-7431-477e-86d5-4fdf667756b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:27.913242 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e947af6-7431-477e-86d5-4fdf667756b8 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:27.913521 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e947af6-7431-477e-86d5-4fdf667756b8 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:27.913790 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e947af6-7431-477e-86d5-4fdf667756b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:27.915048 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e947af6-7431-477e-86d5-4fdf667756b8 None admin] RBAC: Adding query filter params (group.id=30c24407ddbb4d64af64c09b373e5a25, role.id=cdb3aa1bd85b4694964629ff7cf8039b, scope.project.id=1b12f8d03f4f4f4c8b5f37b804de18c5) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:27.915356 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e947af6-7431-477e-86d5-4fdf667756b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:27.917718 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e947af6-7431-477e-86d5-4fdf667756b8 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:27.924328 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 184/369] 10.222.0.15 () {56 vars in 1376 bytes} [Sun Sep 11 10:33:27 2022] GET /identity/v3/role_assignments?group.id=30c24407ddbb4d64af64c09b373e5a25&scope.project.id=1b12f8d03f4f4f4c8b5f37b804de18c5&role.id=cdb3aa1bd85b4694964629ff7cf8039b => generated 260 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:27.925215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e947af6-7431-477e-86d5-4fdf667756b8 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:27.947239 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d66af5e-0526-4fa2-9bdc-3e568178de63 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:27.991920 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8294dd-8341-48fa-9945-10c2c3ca84a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:27.993713 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8294dd-8341-48fa-9945-10c2c3ca84a3 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:27.994808 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8294dd-8341-48fa-9945-10c2c3ca84a3 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:27.995276 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8294dd-8341-48fa-9945-10c2c3ca84a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:27.996234 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8294dd-8341-48fa-9945-10c2c3ca84a3 None admin] RBAC: Adding query filter params (group.id=30c24407ddbb4d64af64c09b373e5a25, role.id=f836487a66c74c05838133c9d0614bbf, scope.project.id=6d140898a8684238bddc9872a32eb94c) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:27.996740 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8294dd-8341-48fa-9945-10c2c3ca84a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:27.998469 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8294dd-8341-48fa-9945-10c2c3ca84a3 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:28.001468 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:28.005462 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 186/370] 10.222.0.15 () {56 vars in 1376 bytes} [Sun Sep 11 10:33:27 2022] GET /identity/v3/role_assignments?group.id=30c24407ddbb4d64af64c09b373e5a25&scope.project.id=6d140898a8684238bddc9872a32eb94c&role.id=f836487a66c74c05838133c9d0614bbf => generated 260 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:28.007610 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8294dd-8341-48fa-9945-10c2c3ca84a3 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:28.061711 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:28.096780 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e947af6-7431-477e-86d5-4fdf667756b8 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:28.137196 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c965628-2091-4c1d-9d9a-fdea3b231c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:28.137826 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c965628-2091-4c1d-9d9a-fdea3b231c12 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:28.138110 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c965628-2091-4c1d-9d9a-fdea3b231c12 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:28.138405 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c965628-2091-4c1d-9d9a-fdea3b231c12 None admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:28.139080 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c965628-2091-4c1d-9d9a-fdea3b231c12 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:28.139372 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c965628-2091-4c1d-9d9a-fdea3b231c12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:28.140811 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c965628-2091-4c1d-9d9a-fdea3b231c12 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:28.150947 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 185/371] 10.222.0.15 () {56 vars in 1111 bytes} [Sun Sep 11 10:33:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:28.157438 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c965628-2091-4c1d-9d9a-fdea3b231c12 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:28.172049 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8294dd-8341-48fa-9945-10c2c3ca84a3 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:28.254361 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf86197-180e-42f0-8ece-9b57eb2b7d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:28.254917 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf86197-180e-42f0-8ece-9b57eb2b7d20 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:28.255197 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf86197-180e-42f0-8ece-9b57eb2b7d20 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:28.255494 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf86197-180e-42f0-8ece-9b57eb2b7d20 None admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:28.256947 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf86197-180e-42f0-8ece-9b57eb2b7d20 None admin] RBAC: Adding query filter params (name=admin) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:28.256947 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf86197-180e-42f0-8ece-9b57eb2b7d20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:28.257940 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf86197-180e-42f0-8ece-9b57eb2b7d20 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:28.270642 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 187/372] 10.222.0.15 () {56 vars in 1099 bytes} [Sun Sep 11 10:33:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:28.284163 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf86197-180e-42f0-8ece-9b57eb2b7d20 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:28.341546 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c965628-2091-4c1d-9d9a-fdea3b231c12 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:28.361058 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:28.394778 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b82575-a102-4636-b634-d076e4ade869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:28.395695 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b82575-a102-4636-b634-d076e4ade869 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:28.396137 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b82575-a102-4636-b634-d076e4ade869 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:28.398506 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b82575-a102-4636-b634-d076e4ade869 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:28.479752 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:28.484855 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf86197-180e-42f0-8ece-9b57eb2b7d20 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:28.494433 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b82575-a102-4636-b634-d076e4ade869 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:28.496352 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b82575-a102-4636-b634-d076e4ade869 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:28.531436 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316eafae-a43b-420b-9996-4dd2a2f6d1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:28.532086 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316eafae-a43b-420b-9996-4dd2a2f6d1fd None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:28.532367 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316eafae-a43b-420b-9996-4dd2a2f6d1fd None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:28.532642 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316eafae-a43b-420b-9996-4dd2a2f6d1fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:28.635646 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316eafae-a43b-420b-9996-4dd2a2f6d1fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:28.636589 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 186/373] 10.222.0.15 () {58 vars in 1282 bytes} [Sun Sep 11 10:33:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 11 10:33:28.637116 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316eafae-a43b-420b-9996-4dd2a2f6d1fd None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:28.638111 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aa02f7-e1a0-4ad0-8303-229259f5256f None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:28.638398 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aa02f7-e1a0-4ad0-8303-229259f5256f None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:28.638663 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aa02f7-e1a0-4ad0-8303-229259f5256f None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:28.641074 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 187/374] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:33:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:33:28.641986 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aa02f7-e1a0-4ad0-8303-229259f5256f None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:28.665115 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:28.771088 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:28.777532 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 188/375] 10.222.0.15 () {58 vars in 1282 bytes} [Sun Sep 11 10:33:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 10:33:28.777902 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aa02f7-e1a0-4ad0-8303-229259f5256f None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:28.778930 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70162e7-a70f-4a0a-910b-a434998d9439 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:28.781584 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70162e7-a70f-4a0a-910b-a434998d9439 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:28.781584 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70162e7-a70f-4a0a-910b-a434998d9439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:28.803753 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-b70162e7-a70f-4a0a-910b-a434998d9439 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:33:28.805465 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-b70162e7-a70f-4a0a-910b-a434998d9439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:33:28.817230 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c02c18-9586-4d25-b934-4556bee05205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:28.818094 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c02c18-9586-4d25-b934-4556bee05205 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:28.818611 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c02c18-9586-4d25-b934-4556bee05205 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:28.819018 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c02c18-9586-4d25-b934-4556bee05205 None admin] PATH_INFO: `/v3/projects/6d140898a8684238bddc9872a32eb94c` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:28.820383 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c02c18-9586-4d25-b934-4556bee05205 None admin] RBAC: Authorizing `identity:get_project(project_id=6d140898a8684238bddc9872a32eb94c)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:28.822232 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c02c18-9586-4d25-b934-4556bee05205 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:28.824465 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 188/376] 10.222.0.15 () {56 vars in 1186 bytes} [Sun Sep 11 10:33:28 2022] GET /identity/v3/projects/6d140898a8684238bddc9872a32eb94c => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:28.825506 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c02c18-9586-4d25-b934-4556bee05205 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:28.886621 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 189/377] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:33:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:33:28.891449 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70162e7-a70f-4a0a-910b-a434998d9439 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:28.942662 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:28.970324 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c02c18-9586-4d25-b934-4556bee05205 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:29.011450 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70162e7-a70f-4a0a-910b-a434998d9439 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:29.021720 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499d2b95-85f1-4401-913e-e1bdf1b843e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:29.022548 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d2b95-85f1-4401-913e-e1bdf1b843e5 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:29.022987 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d2b95-85f1-4401-913e-e1bdf1b843e5 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:29.024004 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d2b95-85f1-4401-913e-e1bdf1b843e5 None admin] PATH_INFO: `/v3/projects/dcf7c8cb1c844da09226c671da5a163e` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:29.028239 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499d2b95-85f1-4401-913e-e1bdf1b843e5 None admin] RBAC: Authorizing `identity:get_project(project_id=dcf7c8cb1c844da09226c671da5a163e)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:29.030053 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499d2b95-85f1-4401-913e-e1bdf1b843e5 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:29.032244 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 189/378] 10.222.0.15 () {56 vars in 1186 bytes} [Sun Sep 11 10:33:28 2022] GET /identity/v3/projects/dcf7c8cb1c844da09226c671da5a163e => generated 344 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:29.037181 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499d2b95-85f1-4401-913e-e1bdf1b843e5 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:29.039970 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:29.055466 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8647d3-a720-42b8-b1af-aa6151070726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:29.055466 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8647d3-a720-42b8-b1af-aa6151070726 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:29.055466 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8647d3-a720-42b8-b1af-aa6151070726 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:29.055466 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8647d3-a720-42b8-b1af-aa6151070726 None admin] PATH_INFO: `/v3/groups/30c24407ddbb4d64af64c09b373e5a25` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:29.056035 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8647d3-a720-42b8-b1af-aa6151070726 None admin] RBAC: Authorizing `identity:get_group(group_id=30c24407ddbb4d64af64c09b373e5a25)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:29.056035 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8647d3-a720-42b8-b1af-aa6151070726 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:29.058254 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 190/379] 10.222.0.15 () {56 vars in 1180 bytes} [Sun Sep 11 10:33:28 2022] GET /identity/v3/groups/30c24407ddbb4d64af64c09b373e5a25 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:29.065000 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8647d3-a720-42b8-b1af-aa6151070726 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:29.182040 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8647d3-a720-42b8-b1af-aa6151070726 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:29.197536 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499d2b95-85f1-4401-913e-e1bdf1b843e5 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:29.218073 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437bd68d-ba9f-4b72-9131-e4a2cd49c192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:29.218779 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437bd68d-ba9f-4b72-9131-e4a2cd49c192 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:29.219062 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437bd68d-ba9f-4b72-9131-e4a2cd49c192 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:29.219334 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437bd68d-ba9f-4b72-9131-e4a2cd49c192 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:29.220116 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437bd68d-ba9f-4b72-9131-e4a2cd49c192 None admin] RBAC: Adding query filter params (group.id=30c24407ddbb4d64af64c09b373e5a25, role.id=cdb3aa1bd85b4694964629ff7cf8039b, scope.project.id=6d140898a8684238bddc9872a32eb94c) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:29.220421 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437bd68d-ba9f-4b72-9131-e4a2cd49c192 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:29.221847 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437bd68d-ba9f-4b72-9131-e4a2cd49c192 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:29.228444 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 191/380] 10.222.0.15 () {56 vars in 1376 bytes} [Sun Sep 11 10:33:29 2022] GET /identity/v3/role_assignments?group.id=30c24407ddbb4d64af64c09b373e5a25&scope.project.id=6d140898a8684238bddc9872a32eb94c&role.id=cdb3aa1bd85b4694964629ff7cf8039b => generated 260 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:29.247387 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f533722b-7750-4a5a-b211-a18a26004691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:29.248933 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f533722b-7750-4a5a-b211-a18a26004691 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:29.249211 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f533722b-7750-4a5a-b211-a18a26004691 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:29.249485 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f533722b-7750-4a5a-b211-a18a26004691 None admin] PATH_INFO: `/v3/groups/dfe4e0ab839a4157aa444073fea888a2` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:29.255705 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f533722b-7750-4a5a-b211-a18a26004691 None admin] RBAC: Authorizing `identity:get_group(group_id=dfe4e0ab839a4157aa444073fea888a2)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:29.257208 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f533722b-7750-4a5a-b211-a18a26004691 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:29.259831 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 190/381] 10.222.0.15 () {56 vars in 1180 bytes} [Sun Sep 11 10:33:29 2022] GET /identity/v3/groups/dfe4e0ab839a4157aa444073fea888a2 => generated 229 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:33:29.265909 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437bd68d-ba9f-4b72-9131-e4a2cd49c192 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:29.352345 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:29.369285 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f533722b-7750-4a5a-b211-a18a26004691 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:29.391548 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:29.400165 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437bd68d-ba9f-4b72-9131-e4a2cd49c192 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:29.436738 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846d8b06-9921-48af-ad2f-46f5e8a15d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:29.437353 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846d8b06-9921-48af-ad2f-46f5e8a15d90 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:29.437639 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846d8b06-9921-48af-ad2f-46f5e8a15d90 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:29.437951 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846d8b06-9921-48af-ad2f-46f5e8a15d90 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:29.438717 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846d8b06-9921-48af-ad2f-46f5e8a15d90 None admin] RBAC: Adding query filter params (group.id=dfe4e0ab839a4157aa444073fea888a2, role.id=2cb7d73f0ea8414296367573f6902340, scope.project.id=dcf7c8cb1c844da09226c671da5a163e) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:29.439017 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846d8b06-9921-48af-ad2f-46f5e8a15d90 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:29.440507 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846d8b06-9921-48af-ad2f-46f5e8a15d90 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:29.449355 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 192/382] 10.222.0.15 () {56 vars in 1376 bytes} [Sun Sep 11 10:33:29 2022] GET /identity/v3/role_assignments?group.id=dfe4e0ab839a4157aa444073fea888a2&scope.project.id=dcf7c8cb1c844da09226c671da5a163e&role.id=2cb7d73f0ea8414296367573f6902340 => generated 260 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:29.507417 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f533722b-7750-4a5a-b211-a18a26004691 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:29.553262 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f93464e-41e6-4891-97e2-d1d4b9206429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:29.554309 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f93464e-41e6-4891-97e2-d1d4b9206429 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:29.555338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f93464e-41e6-4891-97e2-d1d4b9206429 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:29.555849 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f93464e-41e6-4891-97e2-d1d4b9206429 None admin] PATH_INFO: `/v3/roles/member` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:29.568155 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f93464e-41e6-4891-97e2-d1d4b9206429 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:33:29.574524 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-5f93464e-41e6-4891-97e2-d1d4b9206429 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 10:33:29.574524 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:29.574524 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:29.574524 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:29.574524 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:29.574524 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:29.574524 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:29.574524 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:29.574524 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:29.574524 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:29.574524 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:29.574524 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:29.574524 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:33:29.574524 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:33:29.574524 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:33:29.574524 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:33:29.575553 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:33:29.575553 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:33:29.575553 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:33:29.575553 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:33:29.575553 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:29.575553 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:29.575553 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:29.575553 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:29.575553 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:29.575553 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:29.575553 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:29.575553 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:29.575553 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:29.575553 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:29.575553 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:29.575553 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:29.576355 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:29.576355 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:29.576355 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:29.576355 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:29.576355 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:33:29.576355 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:33:29.576355 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:33:29.576355 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:33:29.576355 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:33:29.576355 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:33:29.576355 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 10:33:29.576355 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:33:29.584759 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 191/383] 10.222.0.15 () {56 vars in 1099 bytes} [Sun Sep 11 10:33:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:29.585765 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846d8b06-9921-48af-ad2f-46f5e8a15d90 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:29.730221 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846d8b06-9921-48af-ad2f-46f5e8a15d90 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:29.757415 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:29.778297 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9999ea07-c479-47a6-b6b0-9e14e3b1e433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:29.778910 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999ea07-c479-47a6-b6b0-9e14e3b1e433 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:29.779180 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999ea07-c479-47a6-b6b0-9e14e3b1e433 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:29.779478 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999ea07-c479-47a6-b6b0-9e14e3b1e433 None admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:29.780147 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9999ea07-c479-47a6-b6b0-9e14e3b1e433 None admin] RBAC: Adding query filter params (name=member) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:29.780449 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9999ea07-c479-47a6-b6b0-9e14e3b1e433 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:29.781828 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9999ea07-c479-47a6-b6b0-9e14e3b1e433 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:29.792742 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 193/384] 10.222.0.15 () {56 vars in 1101 bytes} [Sun Sep 11 10:33:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:29.798533 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f93464e-41e6-4891-97e2-d1d4b9206429 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:29.908859 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:29.944318 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f93464e-41e6-4891-97e2-d1d4b9206429 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:29.989433 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d59e24-1b6d-4768-9567-d53e7b920fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:29.990075 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d59e24-1b6d-4768-9567-d53e7b920fbb None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:29.990356 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d59e24-1b6d-4768-9567-d53e7b920fbb None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:29.990686 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d59e24-1b6d-4768-9567-d53e7b920fbb None admin] PATH_INFO: `/v3/groups/30c24407ddbb4d64af64c09b373e5a25` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:29.992365 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d59e24-1b6d-4768-9567-d53e7b920fbb None admin] RBAC: Authorizing `identity:get_group(group_id=30c24407ddbb4d64af64c09b373e5a25)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:29.993832 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d59e24-1b6d-4768-9567-d53e7b920fbb None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:29.995746 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 192/385] 10.222.0.15 () {56 vars in 1180 bytes} [Sun Sep 11 10:33:29 2022] GET /identity/v3/groups/30c24407ddbb4d64af64c09b373e5a25 => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:30.001303 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9999ea07-c479-47a6-b6b0-9e14e3b1e433 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:30.118344 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9999ea07-c479-47a6-b6b0-9e14e3b1e433 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:30.165399 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c1ff39-6299-4e7c-94fb-4cd0823e3792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:30.165971 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c1ff39-6299-4e7c-94fb-4cd0823e3792 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:30.166253 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c1ff39-6299-4e7c-94fb-4cd0823e3792 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:30.166527 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c1ff39-6299-4e7c-94fb-4cd0823e3792 None admin] PATH_INFO: `/v3/projects/1b12f8d03f4f4f4c8b5f37b804de18c5` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:30.167707 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c1ff39-6299-4e7c-94fb-4cd0823e3792 None admin] RBAC: Authorizing `identity:get_project(project_id=1b12f8d03f4f4f4c8b5f37b804de18c5)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:30.169163 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c1ff39-6299-4e7c-94fb-4cd0823e3792 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:30.171124 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 194/386] 10.222.0.15 () {56 vars in 1186 bytes} [Sun Sep 11 10:33:30 2022] GET /identity/v3/projects/1b12f8d03f4f4f4c8b5f37b804de18c5 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:30.176815 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d59e24-1b6d-4768-9567-d53e7b920fbb None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:30.291409 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d59e24-1b6d-4768-9567-d53e7b920fbb None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:30.333552 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b196352-9a89-4403-ae7d-5554c9b7d63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:30.334233 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b196352-9a89-4403-ae7d-5554c9b7d63f None admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:30.334513 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b196352-9a89-4403-ae7d-5554c9b7d63f None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:30.334785 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b196352-9a89-4403-ae7d-5554c9b7d63f None admin] PATH_INFO: `/v3/projects/1b12f8d03f4f4f4c8b5f37b804de18c5/groups/30c24407ddbb4d64af64c09b373e5a25/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:30.337297 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b196352-9a89-4403-ae7d-5554c9b7d63f None admin] RBAC: Authorizing `identity:create_grant(project_id=1b12f8d03f4f4f4c8b5f37b804de18c5, group_id=30c24407ddbb4d64af64c09b373e5a25, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:30.338808 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b196352-9a89-4403-ae7d-5554c9b7d63f None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:30.365833 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 193/387] 10.222.0.15 () {58 vars in 1442 bytes} [Sun Sep 11 10:33:30 2022] PUT /identity/v3/projects/1b12f8d03f4f4f4c8b5f37b804de18c5/groups/30c24407ddbb4d64af64c09b373e5a25/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:33:30.712904 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efef52cb-b0ca-4877-9a9f-2ec508445628 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:30.713638 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efef52cb-b0ca-4877-9a9f-2ec508445628 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:30.714128 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efef52cb-b0ca-4877-9a9f-2ec508445628 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:30.716233 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 195/388] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:33:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:33:30.721609 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bff2ce-3c2c-4da7-90fb-e6d57db95fce None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:30.722133 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bff2ce-3c2c-4da7-90fb-e6d57db95fce None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:30.722637 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bff2ce-3c2c-4da7-90fb-e6d57db95fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:30.732222 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:30.746114 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-d8bff2ce-3c2c-4da7-90fb-e6d57db95fce None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:33:30.747326 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-d8bff2ce-3c2c-4da7-90fb-e6d57db95fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:33:30.843597 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 194/389] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:33:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:33:31.193794 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59edc7b9-98c5-4b06-8553-f738664ee1db None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:31.194339 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59edc7b9-98c5-4b06-8553-f738664ee1db None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:31.194416 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59edc7b9-98c5-4b06-8553-f738664ee1db None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:31.196324 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 196/390] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:33:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:33:31.197040 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bff2ce-3c2c-4da7-90fb-e6d57db95fce None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:31.201206 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ba4d6-3e4b-42e2-9115-75a0e9afed4b None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:31.201489 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ba4d6-3e4b-42e2-9115-75a0e9afed4b None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:31.201759 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ba4d6-3e4b-42e2-9115-75a0e9afed4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:31.230365 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-ab7ba4d6-3e4b-42e2-9115-75a0e9afed4b None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:33:31.232590 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-ab7ba4d6-3e4b-42e2-9115-75a0e9afed4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:33:31.352178 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:31.357583 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bff2ce-3c2c-4da7-90fb-e6d57db95fce None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:31.381693 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 197/391] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:33:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 10:33:31.418470 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8049d6b4-cb86-4428-866b-35d871353eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:31.419112 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8049d6b4-cb86-4428-866b-35d871353eaa None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:31.419393 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8049d6b4-cb86-4428-866b-35d871353eaa None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:31.419692 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8049d6b4-cb86-4428-866b-35d871353eaa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:31.433447 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8049d6b4-cb86-4428-866b-35d871353eaa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:33:31.438242 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-8049d6b4-cb86-4428-866b-35d871353eaa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 10:33:31.438242 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:31.438242 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:31.438242 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:31.438242 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:31.438242 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:31.438242 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:31.438242 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:31.438242 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:31.438242 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:31.438242 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:31.438242 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:31.438242 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:33:31.438242 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:33:31.438242 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:33:31.438242 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:33:31.439145 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:33:31.439145 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:33:31.439145 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:33:31.439145 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:33:31.439145 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:31.439145 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:31.439145 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:31.439145 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:31.439145 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:31.439145 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:31.439145 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:31.439145 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:31.439145 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:31.439145 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:31.439145 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:31.439145 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:31.440375 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:31.440375 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:31.440375 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:31.440375 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:31.440375 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:33:31.440375 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:33:31.440375 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:33:31.440375 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:33:31.440375 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:33:31.440375 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:33:31.440375 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 10:33:31.440375 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:33:31.440375 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 195/392] 10.222.0.15 () {56 vars in 1114 bytes} [Sun Sep 11 10:33:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:31.441531 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:31.449467 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7ba4d6-3e4b-42e2-9115-75a0e9afed4b None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:31.604402 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7ba4d6-3e4b-42e2-9115-75a0e9afed4b None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:31.657537 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d62663b-406a-4e45-98ce-5ec91263e4c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:31.658166 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d62663b-406a-4e45-98ce-5ec91263e4c2 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:31.658445 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d62663b-406a-4e45-98ce-5ec91263e4c2 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:31.658720 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d62663b-406a-4e45-98ce-5ec91263e4c2 None admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:31.659380 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d62663b-406a-4e45-98ce-5ec91263e4c2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:31.659709 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d62663b-406a-4e45-98ce-5ec91263e4c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:31.661072 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d62663b-406a-4e45-98ce-5ec91263e4c2 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:31.676018 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 198/393] 10.222.0.15 () {56 vars in 1111 bytes} [Sun Sep 11 10:33:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:31.682083 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8049d6b4-cb86-4428-866b-35d871353eaa None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:31.705119 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d62663b-406a-4e45-98ce-5ec91263e4c2 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:31.766891 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:31.869423 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8049d6b4-cb86-4428-866b-35d871353eaa None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:31.871989 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d62663b-406a-4e45-98ce-5ec91263e4c2 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:31.923783 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b2ce5a-b0ab-49e4-8108-d1a0fe32d90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:31.925394 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b2ce5a-b0ab-49e4-8108-d1a0fe32d90c None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:31.925668 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b2ce5a-b0ab-49e4-8108-d1a0fe32d90c None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:31.925938 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b2ce5a-b0ab-49e4-8108-d1a0fe32d90c None admin] PATH_INFO: `/v3/groups/30c24407ddbb4d64af64c09b373e5a25` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:31.927220 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b2ce5a-b0ab-49e4-8108-d1a0fe32d90c None admin] RBAC: Authorizing `identity:get_group(group_id=30c24407ddbb4d64af64c09b373e5a25)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:31.928674 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b2ce5a-b0ab-49e4-8108-d1a0fe32d90c None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:31.931231 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 196/394] 10.222.0.15 () {56 vars in 1180 bytes} [Sun Sep 11 10:33:31 2022] GET /identity/v3/groups/30c24407ddbb4d64af64c09b373e5a25 => generated 226 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:31.937088 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b2ce5a-b0ab-49e4-8108-d1a0fe32d90c None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:31.951058 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68a4e5e-0eca-4f9d-8752-e6048bfd31d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:31.951733 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68a4e5e-0eca-4f9d-8752-e6048bfd31d2 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:31.952007 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68a4e5e-0eca-4f9d-8752-e6048bfd31d2 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:31.952273 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68a4e5e-0eca-4f9d-8752-e6048bfd31d2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:31.973929 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68a4e5e-0eca-4f9d-8752-e6048bfd31d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:33:31.985670 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-b68a4e5e-0eca-4f9d-8752-e6048bfd31d2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 10:33:31.985670 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:31.985670 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:31.985670 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:31.985670 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:31.985670 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:31.985670 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:31.985670 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:31.985670 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:31.985670 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:31.985670 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:31.985670 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:31.985670 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:33:31.985670 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:33:31.985670 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:33:31.985670 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:33:31.986616 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:33:31.986616 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:33:31.986616 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:33:31.986616 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:33:31.986616 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:31.986616 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:31.986616 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:31.986616 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:31.986616 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:31.986616 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:31.986616 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:31.986616 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:31.986616 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:31.986616 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:31.986616 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:31.986616 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:31.987410 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:31.987410 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:31.987410 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:31.987410 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:31.987410 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:33:31.987410 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:33:31.987410 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:33:31.987410 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:33:31.987410 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:33:31.987410 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:33:31.987410 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 10:33:31.987410 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:33:31.996080 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 199/395] 10.222.0.15 () {56 vars in 1099 bytes} [Sun Sep 11 10:33:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:32.001781 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68a4e5e-0eca-4f9d-8752-e6048bfd31d2 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:32.043459 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:32.119056 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b2ce5a-b0ab-49e4-8108-d1a0fe32d90c None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:32.181013 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b540381-bae5-469f-b225-567356efa641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:32.181811 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68a4e5e-0eca-4f9d-8752-e6048bfd31d2 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:32.182252 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b540381-bae5-469f-b225-567356efa641 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:32.182739 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b540381-bae5-469f-b225-567356efa641 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:32.183359 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b540381-bae5-469f-b225-567356efa641 None admin] PATH_INFO: `/v3/projects/1b12f8d03f4f4f4c8b5f37b804de18c5` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:32.185541 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b540381-bae5-469f-b225-567356efa641 None admin] RBAC: Authorizing `identity:get_project(project_id=1b12f8d03f4f4f4c8b5f37b804de18c5)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:32.186993 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b540381-bae5-469f-b225-567356efa641 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:32.192511 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 197/396] 10.222.0.15 () {56 vars in 1186 bytes} [Sun Sep 11 10:33:31 2022] GET /identity/v3/projects/1b12f8d03f4f4f4c8b5f37b804de18c5 => generated 298 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:32.192685 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:32.192763 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264fb1d7-418e-4e43-a6af-5007d0eba8f1 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:32.192763 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264fb1d7-418e-4e43-a6af-5007d0eba8f1 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:32.192763 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264fb1d7-418e-4e43-a6af-5007d0eba8f1 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:32.192763 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 198/397] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:33:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:33:32.192763 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca32368-428d-4605-9473-16f1ccb18213 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:32.192763 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca32368-428d-4605-9473-16f1ccb18213 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:32.193122 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca32368-428d-4605-9473-16f1ccb18213 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:32.194417 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 199/398] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:33:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:33:32.200050 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7841c88-3008-4d89-9084-5e12889601bd None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:32.200339 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7841c88-3008-4d89-9084-5e12889601bd None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:32.200612 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7841c88-3008-4d89-9084-5e12889601bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:32.218713 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-e7841c88-3008-4d89-9084-5e12889601bd None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:33:32.219926 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-e7841c88-3008-4d89-9084-5e12889601bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:33:32.224570 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8c075b-9889-4055-8bed-6bacf7e70550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:32.225137 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8c075b-9889-4055-8bed-6bacf7e70550 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:32.225400 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8c075b-9889-4055-8bed-6bacf7e70550 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:32.225677 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8c075b-9889-4055-8bed-6bacf7e70550 None admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:32.226392 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8c075b-9889-4055-8bed-6bacf7e70550 None admin] RBAC: Adding query filter params (name=member) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:32.227258 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8c075b-9889-4055-8bed-6bacf7e70550 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:32.228752 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8c075b-9889-4055-8bed-6bacf7e70550 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:32.239504 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 200/399] 10.222.0.15 () {56 vars in 1101 bytes} [Sun Sep 11 10:33:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:33:32.239504 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8c075b-9889-4055-8bed-6bacf7e70550 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:32.297345 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 200/400] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:33:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:33:32.301484 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c2ae38-0f19-49fd-b21e-28a405988d08 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:32.302189 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c2ae38-0f19-49fd-b21e-28a405988d08 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:32.302673 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c2ae38-0f19-49fd-b21e-28a405988d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:32.331027 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-46c2ae38-0f19-49fd-b21e-28a405988d08 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:33:32.332157 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-46c2ae38-0f19-49fd-b21e-28a405988d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:33:32.366744 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8c075b-9889-4055-8bed-6bacf7e70550 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:32.393981 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:32.411791 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 201/401] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:33:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:33:32.412760 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c2ae38-0f19-49fd-b21e-28a405988d08 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:32.413579 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6badb5-1739-42e4-b381-edfd22b1ad6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:32.415023 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6badb5-1739-42e4-b381-edfd22b1ad6d None admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:32.415948 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6badb5-1739-42e4-b381-edfd22b1ad6d None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:32.416431 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6badb5-1739-42e4-b381-edfd22b1ad6d None admin] PATH_INFO: `/v3/projects/1b12f8d03f4f4f4c8b5f37b804de18c5/groups/30c24407ddbb4d64af64c09b373e5a25/roles/cdb3aa1bd85b4694964629ff7cf8039b` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:32.418887 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6badb5-1739-42e4-b381-edfd22b1ad6d None admin] RBAC: Authorizing `identity:create_grant(project_id=1b12f8d03f4f4f4c8b5f37b804de18c5, group_id=30c24407ddbb4d64af64c09b373e5a25, role_id=cdb3aa1bd85b4694964629ff7cf8039b)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:32.421507 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6badb5-1739-42e4-b381-edfd22b1ad6d None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:32.443601 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 201/402] 10.222.0.15 () {58 vars in 1442 bytes} [Sun Sep 11 10:33:32 2022] PUT /identity/v3/projects/1b12f8d03f4f4f4c8b5f37b804de18c5/groups/30c24407ddbb4d64af64c09b373e5a25/roles/cdb3aa1bd85b4694964629ff7cf8039b => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:33:32.541129 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c2ae38-0f19-49fd-b21e-28a405988d08 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:32.581689 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860fc437-b6c1-47a4-b200-15e537936d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:32.582721 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860fc437-b6c1-47a4-b200-15e537936d8e None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:32.583226 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860fc437-b6c1-47a4-b200-15e537936d8e None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:32.583720 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860fc437-b6c1-47a4-b200-15e537936d8e None admin] PATH_INFO: `/v3/groups/30c24407ddbb4d64af64c09b373e5a25` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:32.585097 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860fc437-b6c1-47a4-b200-15e537936d8e None admin] RBAC: Authorizing `identity:get_group(group_id=30c24407ddbb4d64af64c09b373e5a25)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:32.587562 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860fc437-b6c1-47a4-b200-15e537936d8e None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:32.590193 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 202/403] 10.222.0.15 () {56 vars in 1180 bytes} [Sun Sep 11 10:33:32 2022] GET /identity/v3/groups/30c24407ddbb4d64af64c09b373e5a25 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:32.595350 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6badb5-1739-42e4-b381-edfd22b1ad6d None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:32.623219 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860fc437-b6c1-47a4-b200-15e537936d8e None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:32.639249 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:32.715866 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:32.741211 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6badb5-1739-42e4-b381-edfd22b1ad6d None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:32.788767 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860fc437-b6c1-47a4-b200-15e537936d8e None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:32.792209 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a87701f-f787-4661-a540-9072651575f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:32.793186 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a87701f-f787-4661-a540-9072651575f3 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:32.793468 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a87701f-f787-4661-a540-9072651575f3 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:32.793741 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a87701f-f787-4661-a540-9072651575f3 None admin] PATH_INFO: `/v3/projects/6d140898a8684238bddc9872a32eb94c` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:32.795890 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a87701f-f787-4661-a540-9072651575f3 None admin] RBAC: Authorizing `identity:get_project(project_id=6d140898a8684238bddc9872a32eb94c)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:32.797347 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a87701f-f787-4661-a540-9072651575f3 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:32.799155 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 202/404] 10.222.0.15 () {56 vars in 1186 bytes} [Sun Sep 11 10:33:32 2022] GET /identity/v3/projects/6d140898a8684238bddc9872a32eb94c => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:32.804257 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a87701f-f787-4661-a540-9072651575f3 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:32.843519 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb97550-8aad-402d-8d5c-fdf59dd8c66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:32.844514 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb97550-8aad-402d-8d5c-fdf59dd8c66a None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:32.844987 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb97550-8aad-402d-8d5c-fdf59dd8c66a None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:32.845451 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb97550-8aad-402d-8d5c-fdf59dd8c66a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:32.862006 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb97550-8aad-402d-8d5c-fdf59dd8c66a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:33:32.869284 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-2eb97550-8aad-402d-8d5c-fdf59dd8c66a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 10:33:32.869284 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:32.869284 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:32.869284 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:32.869284 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:32.869284 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:32.869284 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:32.869284 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:32.869284 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:32.869284 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:32.869284 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:32.869284 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:32.869284 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:33:32.869284 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:33:32.869284 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:33:32.869284 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:33:32.870603 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:33:32.870603 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:33:32.870603 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:33:32.870603 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:33:32.870603 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:32.870603 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:32.870603 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:32.870603 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:32.870603 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:32.870603 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:32.870603 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:32.870603 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:32.870603 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:32.870603 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:32.870603 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:32.870603 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:32.871399 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:32.871399 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:32.871399 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:32.871399 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:32.871399 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:33:32.871399 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:33:32.871399 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:33:32.871399 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:33:32.871399 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:33:32.871399 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:33:32.871399 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 10:33:32.871399 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:33:32.874423 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 203/405] 10.222.0.15 () {56 vars in 1114 bytes} [Sun Sep 11 10:33:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 10:33:32.875259 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb97550-8aad-402d-8d5c-fdf59dd8c66a None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:32.921782 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:32.932055 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a87701f-f787-4661-a540-9072651575f3 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:32.981987 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332e0db6-c60b-46a6-a60c-6444e960a954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:32.982964 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332e0db6-c60b-46a6-a60c-6444e960a954 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:32.983453 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332e0db6-c60b-46a6-a60c-6444e960a954 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:32.983918 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332e0db6-c60b-46a6-a60c-6444e960a954 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:32.998510 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332e0db6-c60b-46a6-a60c-6444e960a954 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:33:32.998775 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb97550-8aad-402d-8d5c-fdf59dd8c66a None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:33.005762 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-332e0db6-c60b-46a6-a60c-6444e960a954 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 10:33:33.005762 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:33.005762 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:33.005762 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:33.005762 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:33.005762 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:33.005762 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:33.005762 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:33.005762 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:33.005762 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:33.005762 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:33.005762 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:33.005762 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:33:33.005762 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:33:33.005762 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:33:33.005762 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:33:33.006713 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:33:33.006713 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:33:33.006713 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:33:33.006713 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:33:33.006713 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:33.006713 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:33.006713 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:33.006713 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:33.006713 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:33.006713 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:33.006713 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:33.006713 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:33.006713 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:33.006713 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:33.006713 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:33.006713 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:33.007559 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:33.007559 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:33.007559 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:33.007559 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:33.007559 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:33:33.007559 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:33:33.007559 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:33:33.007559 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:33:33.007559 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:33:33.007559 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:33:33.007559 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 10:33:33.007559 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:33:33.011696 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 203/406] 10.222.0.15 () {56 vars in 1096 bytes} [Sun Sep 11 10:33:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:33.012829 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332e0db6-c60b-46a6-a60c-6444e960a954 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:33.030912 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:33.055586 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859bec44-9aef-48d9-af23-a9f4a830fde5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:33.056892 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859bec44-9aef-48d9-af23-a9f4a830fde5 None admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:33.057282 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859bec44-9aef-48d9-af23-a9f4a830fde5 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:33.057662 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859bec44-9aef-48d9-af23-a9f4a830fde5 None admin] PATH_INFO: `/v3/projects/6d140898a8684238bddc9872a32eb94c/groups/30c24407ddbb4d64af64c09b373e5a25/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:33.060394 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859bec44-9aef-48d9-af23-a9f4a830fde5 None admin] RBAC: Authorizing `identity:create_grant(project_id=6d140898a8684238bddc9872a32eb94c, group_id=30c24407ddbb4d64af64c09b373e5a25, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:33.061870 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859bec44-9aef-48d9-af23-a9f4a830fde5 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:33.087612 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 204/407] 10.222.0.15 () {58 vars in 1442 bytes} [Sun Sep 11 10:33:32 2022] PUT /identity/v3/projects/6d140898a8684238bddc9872a32eb94c/groups/30c24407ddbb4d64af64c09b373e5a25/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:33:33.088816 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859bec44-9aef-48d9-af23-a9f4a830fde5 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:33.168034 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332e0db6-c60b-46a6-a60c-6444e960a954 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:33.206394 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3cfcc8-0f4c-4fb7-a209-c217b77c7636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:33.207016 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3cfcc8-0f4c-4fb7-a209-c217b77c7636 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:33.207294 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3cfcc8-0f4c-4fb7-a209-c217b77c7636 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:33.207595 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3cfcc8-0f4c-4fb7-a209-c217b77c7636 None admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:33.208262 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3cfcc8-0f4c-4fb7-a209-c217b77c7636 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:33.208560 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3cfcc8-0f4c-4fb7-a209-c217b77c7636 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:33.209955 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3cfcc8-0f4c-4fb7-a209-c217b77c7636 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:33.219348 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 204/408] 10.222.0.15 () {56 vars in 1111 bytes} [Sun Sep 11 10:33:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:33.220493 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b15779-ee60-451e-a26f-8b7e6395dc05 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:33.220780 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b15779-ee60-451e-a26f-8b7e6395dc05 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:33.221158 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b15779-ee60-451e-a26f-8b7e6395dc05 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:33.227495 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 205/409] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:33:33 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:33:33.228395 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b15779-ee60-451e-a26f-8b7e6395dc05 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:33.244526 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859bec44-9aef-48d9-af23-a9f4a830fde5 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:33.253520 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:33.292579 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1877116b-2453-41b8-8081-d6f885389600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:33.293260 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1877116b-2453-41b8-8081-d6f885389600 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:33.293543 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1877116b-2453-41b8-8081-d6f885389600 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:33.293867 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1877116b-2453-41b8-8081-d6f885389600 None admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:33.294764 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1877116b-2453-41b8-8081-d6f885389600 None admin] RBAC: Adding query filter params (name=admin) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:33.295663 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1877116b-2453-41b8-8081-d6f885389600 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:33.296958 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1877116b-2453-41b8-8081-d6f885389600 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:33.307379 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 205/410] 10.222.0.15 () {56 vars in 1099 bytes} [Sun Sep 11 10:33:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:33.308966 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea8845a-37d6-4910-9288-2aa5cbf85578 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:33.308966 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea8845a-37d6-4910-9288-2aa5cbf85578 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:33.309145 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea8845a-37d6-4910-9288-2aa5cbf85578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:33.333277 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-cea8845a-37d6-4910-9288-2aa5cbf85578 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:33:33.334473 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-cea8845a-37d6-4910-9288-2aa5cbf85578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:33:33.368474 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b15779-ee60-451e-a26f-8b7e6395dc05 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:33.374931 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:33.406399 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96914507-3069-490b-8deb-8091fcb86e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:33.407391 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96914507-3069-490b-8deb-8091fcb86e58 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:33.407900 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96914507-3069-490b-8deb-8091fcb86e58 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:33.408319 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96914507-3069-490b-8deb-8091fcb86e58 None admin] PATH_INFO: `/v3/groups/30c24407ddbb4d64af64c09b373e5a25` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:33.409561 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96914507-3069-490b-8deb-8091fcb86e58 None admin] RBAC: Authorizing `identity:get_group(group_id=30c24407ddbb4d64af64c09b373e5a25)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:33.411154 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96914507-3069-490b-8deb-8091fcb86e58 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:33.417553 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 206/412] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:33:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:33:33.417553 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea8845a-37d6-4910-9288-2aa5cbf85578 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:33.417916 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 206/412] 10.222.0.15 () {56 vars in 1180 bytes} [Sun Sep 11 10:33:33 2022] GET /identity/v3/groups/30c24407ddbb4d64af64c09b373e5a25 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:33.418950 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96914507-3069-490b-8deb-8091fcb86e58 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:33.567449 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:33.593506 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96914507-3069-490b-8deb-8091fcb86e58 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:33.606599 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea8845a-37d6-4910-9288-2aa5cbf85578 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:33.657777 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d780eb7-8ffe-4e4b-91ce-de99738bcdbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:33.658428 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d780eb7-8ffe-4e4b-91ce-de99738bcdbf None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:33.658709 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d780eb7-8ffe-4e4b-91ce-de99738bcdbf None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:33.658987 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d780eb7-8ffe-4e4b-91ce-de99738bcdbf None admin] PATH_INFO: `/v3/projects/6d140898a8684238bddc9872a32eb94c` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:33.660643 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d780eb7-8ffe-4e4b-91ce-de99738bcdbf None admin] RBAC: Authorizing `identity:get_project(project_id=6d140898a8684238bddc9872a32eb94c)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:33.662082 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d780eb7-8ffe-4e4b-91ce-de99738bcdbf None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:33.664000 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 207/413] 10.222.0.15 () {56 vars in 1186 bytes} [Sun Sep 11 10:33:33 2022] GET /identity/v3/projects/6d140898a8684238bddc9872a32eb94c => generated 302 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:33.669636 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d780eb7-8ffe-4e4b-91ce-de99738bcdbf None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:33.673028 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc3c1b3-2206-4ea8-b2b5-30a9aaa1104c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:33.674846 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc3c1b3-2206-4ea8-b2b5-30a9aaa1104c None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:33.674846 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc3c1b3-2206-4ea8-b2b5-30a9aaa1104c None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:33.675098 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc3c1b3-2206-4ea8-b2b5-30a9aaa1104c None admin] PATH_INFO: `/v3/groups/dfe4e0ab839a4157aa444073fea888a2` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:33.680697 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc3c1b3-2206-4ea8-b2b5-30a9aaa1104c None admin] RBAC: Authorizing `identity:get_group(group_id=dfe4e0ab839a4157aa444073fea888a2)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:33.682153 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc3c1b3-2206-4ea8-b2b5-30a9aaa1104c None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:33.686180 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 207/414] 10.222.0.15 () {56 vars in 1180 bytes} [Sun Sep 11 10:33:33 2022] GET /identity/v3/groups/dfe4e0ab839a4157aa444073fea888a2 => generated 229 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:33.689812 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc3c1b3-2206-4ea8-b2b5-30a9aaa1104c None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:33.755414 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:33.816039 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d780eb7-8ffe-4e4b-91ce-de99738bcdbf None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:33.816774 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc3c1b3-2206-4ea8-b2b5-30a9aaa1104c None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:33.855716 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2364744e-2c6b-4258-a2bb-d17848edcdd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:33.856275 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364744e-2c6b-4258-a2bb-d17848edcdd7 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:33.856582 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364744e-2c6b-4258-a2bb-d17848edcdd7 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:33.857058 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364744e-2c6b-4258-a2bb-d17848edcdd7 None admin] PATH_INFO: `/v3/projects/dcf7c8cb1c844da09226c671da5a163e` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:33.858072 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2364744e-2c6b-4258-a2bb-d17848edcdd7 None admin] RBAC: Authorizing `identity:get_project(project_id=dcf7c8cb1c844da09226c671da5a163e)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:33.859523 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2364744e-2c6b-4258-a2bb-d17848edcdd7 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:33.862226 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6461c510-c8cc-4f97-b7f0-328dd871f3dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:33.863142 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6461c510-c8cc-4f97-b7f0-328dd871f3dd None admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:33.863659 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6461c510-c8cc-4f97-b7f0-328dd871f3dd None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:33.864124 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6461c510-c8cc-4f97-b7f0-328dd871f3dd None admin] PATH_INFO: `/v3/projects/6d140898a8684238bddc9872a32eb94c/groups/30c24407ddbb4d64af64c09b373e5a25/roles/cdb3aa1bd85b4694964629ff7cf8039b` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:33.866124 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 208/415] 10.222.0.15 () {56 vars in 1186 bytes} [Sun Sep 11 10:33:33 2022] GET /identity/v3/projects/dcf7c8cb1c844da09226c671da5a163e => generated 344 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:33.866708 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6461c510-c8cc-4f97-b7f0-328dd871f3dd None admin] RBAC: Authorizing `identity:create_grant(project_id=6d140898a8684238bddc9872a32eb94c, group_id=30c24407ddbb4d64af64c09b373e5a25, role_id=cdb3aa1bd85b4694964629ff7cf8039b)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:33.867293 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2364744e-2c6b-4258-a2bb-d17848edcdd7 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:33.869188 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6461c510-c8cc-4f97-b7f0-328dd871f3dd None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:33.893632 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 208/416] 10.222.0.15 () {58 vars in 1442 bytes} [Sun Sep 11 10:33:33 2022] PUT /identity/v3/projects/6d140898a8684238bddc9872a32eb94c/groups/30c24407ddbb4d64af64c09b373e5a25/roles/cdb3aa1bd85b4694964629ff7cf8039b => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 10:33:33.895042 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6461c510-c8cc-4f97-b7f0-328dd871f3dd None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:33.931586 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:33.987394 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2364744e-2c6b-4258-a2bb-d17848edcdd7 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:33.996470 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:34.027206 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a37569-1fbf-4b33-b831-b585a0ddf19e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:34.028178 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a37569-1fbf-4b33-b831-b585a0ddf19e None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:34.028677 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a37569-1fbf-4b33-b831-b585a0ddf19e None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:34.029072 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a37569-1fbf-4b33-b831-b585a0ddf19e None admin] PATH_INFO: `/v3/roles/member` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:34.042696 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a37569-1fbf-4b33-b831-b585a0ddf19e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:33:34.048047 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-47a37569-1fbf-4b33-b831-b585a0ddf19e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 10:33:34.048047 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:34.048047 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:34.048047 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:34.048047 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:34.048047 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:34.048047 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:34.048047 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:34.048047 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:34.048047 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:34.048047 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:34.048047 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:34.048047 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:33:34.048047 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:33:34.048047 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:33:34.048047 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:33:34.049037 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:33:34.049037 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:33:34.049037 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:33:34.049037 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:33:34.049037 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:34.049037 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:34.049037 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:34.049037 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:34.049037 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:34.049037 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:34.049037 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:34.049037 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:34.049037 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:34.049037 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:34.049037 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:34.049037 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:34.049826 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:34.049826 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:34.049826 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:34.049826 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:34.049826 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:33:34.049826 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:33:34.049826 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:33:34.049826 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:33:34.049826 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:33:34.049826 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:33:34.049826 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 10:33:34.049826 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:33:34.052689 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 209/417] 10.222.0.15 () {56 vars in 1099 bytes} [Sun Sep 11 10:33:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:34.062174 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a37569-1fbf-4b33-b831-b585a0ddf19e None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:34.104684 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6461c510-c8cc-4f97-b7f0-328dd871f3dd None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:34.146559 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b78e04-3a84-4244-accf-9daec6ae7b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:34.147157 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b78e04-3a84-4244-accf-9daec6ae7b36 None admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:34.147447 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b78e04-3a84-4244-accf-9daec6ae7b36 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:34.147862 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b78e04-3a84-4244-accf-9daec6ae7b36 None admin] PATH_INFO: `/v3/projects/dcf7c8cb1c844da09226c671da5a163e/groups/dfe4e0ab839a4157aa444073fea888a2/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:34.150976 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b78e04-3a84-4244-accf-9daec6ae7b36 None admin] RBAC: Authorizing `identity:create_grant(project_id=dcf7c8cb1c844da09226c671da5a163e, group_id=dfe4e0ab839a4157aa444073fea888a2, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:34.153108 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b78e04-3a84-4244-accf-9daec6ae7b36 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:34.170783 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 209/418] 10.222.0.15 () {58 vars in 1442 bytes} [Sun Sep 11 10:33:33 2022] PUT /identity/v3/projects/dcf7c8cb1c844da09226c671da5a163e/groups/dfe4e0ab839a4157aa444073fea888a2/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:33:34.242946 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a37569-1fbf-4b33-b831-b585a0ddf19e None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:34.287479 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187a9d8c-2e50-4e51-8452-71bc18a030d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:34.287479 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187a9d8c-2e50-4e51-8452-71bc18a030d8 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:34.287479 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187a9d8c-2e50-4e51-8452-71bc18a030d8 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:34.287479 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187a9d8c-2e50-4e51-8452-71bc18a030d8 None admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:34.287879 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187a9d8c-2e50-4e51-8452-71bc18a030d8 None admin] RBAC: Adding query filter params (name=member) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:34.287879 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187a9d8c-2e50-4e51-8452-71bc18a030d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:34.287879 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187a9d8c-2e50-4e51-8452-71bc18a030d8 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:34.298282 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 210/419] 10.222.0.15 () {56 vars in 1101 bytes} [Sun Sep 11 10:33:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:34.304043 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b78e04-3a84-4244-accf-9daec6ae7b36 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:34.407114 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:34.433596 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b78e04-3a84-4244-accf-9daec6ae7b36 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:34.469951 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f64af50-98eb-4697-95ee-a66e0cfe605c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:34.470572 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f64af50-98eb-4697-95ee-a66e0cfe605c None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:34.470850 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f64af50-98eb-4697-95ee-a66e0cfe605c None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:34.471121 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f64af50-98eb-4697-95ee-a66e0cfe605c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:34.562685 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f64af50-98eb-4697-95ee-a66e0cfe605c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:34.564177 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f64af50-98eb-4697-95ee-a66e0cfe605c None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:34.649331 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:34.682434 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 210/420] 10.222.0.15 () {58 vars in 1282 bytes} [Sun Sep 11 10:33:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 11 10:33:34.687993 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187a9d8c-2e50-4e51-8452-71bc18a030d8 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:34.719014 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:34.805886 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187a9d8c-2e50-4e51-8452-71bc18a030d8 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:34.850649 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e811fe6e-ad21-4a2e-a13d-796531f8e5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:34.851280 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e811fe6e-ad21-4a2e-a13d-796531f8e5f2 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:34.851967 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e811fe6e-ad21-4a2e-a13d-796531f8e5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:34.852242 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e811fe6e-ad21-4a2e-a13d-796531f8e5f2 None admin] PATH_INFO: `/v3/projects/1b12f8d03f4f4f4c8b5f37b804de18c5` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:34.853514 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e811fe6e-ad21-4a2e-a13d-796531f8e5f2 None admin] RBAC: Authorizing `identity:get_project(project_id=1b12f8d03f4f4f4c8b5f37b804de18c5)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:34.854952 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e811fe6e-ad21-4a2e-a13d-796531f8e5f2 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:34.856964 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 211/421] 10.222.0.15 () {56 vars in 1186 bytes} [Sun Sep 11 10:33:34 2022] GET /identity/v3/projects/1b12f8d03f4f4f4c8b5f37b804de18c5 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:34.862413 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f64af50-98eb-4697-95ee-a66e0cfe605c None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:34.974936 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f64af50-98eb-4697-95ee-a66e0cfe605c None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:35.010955 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb48cdb-d3db-4845-a6a8-59fbd8478720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:35.011623 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb48cdb-d3db-4845-a6a8-59fbd8478720 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:35.011902 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb48cdb-d3db-4845-a6a8-59fbd8478720 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:35.012172 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb48cdb-d3db-4845-a6a8-59fbd8478720 None admin] PATH_INFO: `/v3/groups/30c24407ddbb4d64af64c09b373e5a25` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:35.013274 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb48cdb-d3db-4845-a6a8-59fbd8478720 None admin] RBAC: Authorizing `identity:get_group(group_id=30c24407ddbb4d64af64c09b373e5a25)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:35.014713 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb48cdb-d3db-4845-a6a8-59fbd8478720 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:35.016314 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 211/422] 10.222.0.15 () {56 vars in 1180 bytes} [Sun Sep 11 10:33:34 2022] GET /identity/v3/groups/30c24407ddbb4d64af64c09b373e5a25 => generated 226 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:35.022176 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e811fe6e-ad21-4a2e-a13d-796531f8e5f2 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:35.029094 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbe4896-7015-40f0-b89b-6bf1d7685ebc None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:35.029385 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbe4896-7015-40f0-b89b-6bf1d7685ebc None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:35.029664 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbe4896-7015-40f0-b89b-6bf1d7685ebc None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:35.031241 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 212/423] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:33:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 10:33:35.036369 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2950f333-f725-458d-9970-1336932ed8a7 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:35.036652 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2950f333-f725-458d-9970-1336932ed8a7 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:35.036922 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2950f333-f725-458d-9970-1336932ed8a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:35.055720 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-2950f333-f725-458d-9970-1336932ed8a7 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:33:35.056927 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-2950f333-f725-458d-9970-1336932ed8a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:33:35.111416 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:35.171674 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 213/424] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:33:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 10:33:35.197806 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:35.204283 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e811fe6e-ad21-4a2e-a13d-796531f8e5f2 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:35.264448 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532098ff-2771-423b-b15a-fb5baff580ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:35.265456 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532098ff-2771-423b-b15a-fb5baff580ba None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:35.265942 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532098ff-2771-423b-b15a-fb5baff580ba None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:35.266417 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532098ff-2771-423b-b15a-fb5baff580ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:35.267386 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532098ff-2771-423b-b15a-fb5baff580ba None admin] RBAC: Adding query filter params (group.id=30c24407ddbb4d64af64c09b373e5a25, role.id=f836487a66c74c05838133c9d0614bbf, scope.project.id=1b12f8d03f4f4f4c8b5f37b804de18c5) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:35.267961 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532098ff-2771-423b-b15a-fb5baff580ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:35.269662 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532098ff-2771-423b-b15a-fb5baff580ba None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:35.280482 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 212/425] 10.222.0.15 () {56 vars in 1376 bytes} [Sun Sep 11 10:33:35 2022] GET /identity/v3/role_assignments?group.id=30c24407ddbb4d64af64c09b373e5a25&scope.project.id=1b12f8d03f4f4f4c8b5f37b804de18c5&role.id=f836487a66c74c05838133c9d0614bbf => generated 611 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:35.479068 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2950f333-f725-458d-9970-1336932ed8a7 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:35.600392 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2950f333-f725-458d-9970-1336932ed8a7 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:35.652422 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edabe28-916f-4214-a048-9dbd1047007f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:35.653148 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edabe28-916f-4214-a048-9dbd1047007f None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:35.653433 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edabe28-916f-4214-a048-9dbd1047007f None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:35.653843 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edabe28-916f-4214-a048-9dbd1047007f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:35.667954 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edabe28-916f-4214-a048-9dbd1047007f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:33:35.673265 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-3edabe28-916f-4214-a048-9dbd1047007f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 10:33:35.673265 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:35.673265 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:35.673265 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:35.673265 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:35.673265 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:35.673265 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:35.673265 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:35.673265 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:35.673265 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:35.673265 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:35.673265 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:35.673265 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:33:35.673265 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:33:35.673265 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:33:35.673265 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:33:35.674150 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:33:35.674150 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:33:35.674150 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:33:35.674150 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:33:35.674150 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:35.674150 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:35.674150 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:35.674150 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:35.674150 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:35.674150 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:35.674150 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:35.674150 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:35.674150 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:35.674150 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:35.674150 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:35.674150 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:35.674962 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:35.674962 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:35.674962 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:35.674962 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:35.674962 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:33:35.674962 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:33:35.674962 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:33:35.674962 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:33:35.674962 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:33:35.674962 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:33:35.674962 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 10:33:35.674962 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:33:35.677526 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 214/426] 10.222.0.15 () {56 vars in 1114 bytes} [Sun Sep 11 10:33:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:35.684110 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532098ff-2771-423b-b15a-fb5baff580ba None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:35.762285 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfcfde0-a83b-4a36-a01d-ac7ee9aeb89a None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:35.762931 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfcfde0-a83b-4a36-a01d-ac7ee9aeb89a None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:35.763404 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfcfde0-a83b-4a36-a01d-ac7ee9aeb89a None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:35.765824 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 215/427] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:33:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 10:33:35.770920 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22a5f51-909f-42ae-8812-caa38efe5942 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:35.771486 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22a5f51-909f-42ae-8812-caa38efe5942 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:35.771994 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22a5f51-909f-42ae-8812-caa38efe5942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:35.797556 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-b22a5f51-909f-42ae-8812-caa38efe5942 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:33:35.799130 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-b22a5f51-909f-42ae-8812-caa38efe5942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:33:35.803258 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532098ff-2771-423b-b15a-fb5baff580ba None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:35.832385 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:35.842487 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c645f93c-9897-4e8b-92f7-764b795e9361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:35.843214 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c645f93c-9897-4e8b-92f7-764b795e9361 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:35.843521 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c645f93c-9897-4e8b-92f7-764b795e9361 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:35.843796 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c645f93c-9897-4e8b-92f7-764b795e9361 None admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:35.844458 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c645f93c-9897-4e8b-92f7-764b795e9361 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:35.844761 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c645f93c-9897-4e8b-92f7-764b795e9361 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:35.846126 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c645f93c-9897-4e8b-92f7-764b795e9361 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:35.856766 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 213/428] 10.222.0.15 () {56 vars in 1111 bytes} [Sun Sep 11 10:33:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:35.862300 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c645f93c-9897-4e8b-92f7-764b795e9361 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:35.887729 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 216/429] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:33:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:33:35.889059 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:35.972548 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c645f93c-9897-4e8b-92f7-764b795e9361 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:36.010412 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f9ccf4-089e-4afb-954f-9c38a711e75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:36.011356 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9ccf4-089e-4afb-954f-9c38a711e75d None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:36.011899 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9ccf4-089e-4afb-954f-9c38a711e75d None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:36.012370 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9ccf4-089e-4afb-954f-9c38a711e75d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:36.087560 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f9ccf4-089e-4afb-954f-9c38a711e75d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:36.089853 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f9ccf4-089e-4afb-954f-9c38a711e75d None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:36.114908 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:36.196790 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22a5f51-909f-42ae-8812-caa38efe5942 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:36.206798 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 214/430] 10.222.0.15 () {58 vars in 1282 bytes} [Sun Sep 11 10:33:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 10:33:36.212826 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f9ccf4-089e-4afb-954f-9c38a711e75d None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:36.326920 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f9ccf4-089e-4afb-954f-9c38a711e75d None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:36.333253 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22a5f51-909f-42ae-8812-caa38efe5942 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:36.354547 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:36.376229 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3804cc67-0abc-45a3-828c-55657c7df00c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:36.377258 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3804cc67-0abc-45a3-828c-55657c7df00c None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:36.377765 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3804cc67-0abc-45a3-828c-55657c7df00c None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:36.378321 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3804cc67-0abc-45a3-828c-55657c7df00c None admin] PATH_INFO: `/v3/projects/1b12f8d03f4f4f4c8b5f37b804de18c5` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:36.379769 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3804cc67-0abc-45a3-828c-55657c7df00c None admin] RBAC: Authorizing `identity:get_project(project_id=1b12f8d03f4f4f4c8b5f37b804de18c5)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:36.381789 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3804cc67-0abc-45a3-828c-55657c7df00c None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:36.383484 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70471e59-21da-461f-932b-54d86c4e7c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:36.384381 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70471e59-21da-461f-932b-54d86c4e7c86 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:36.384497 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 215/431] 10.222.0.15 () {56 vars in 1186 bytes} [Sun Sep 11 10:33:36 2022] GET /identity/v3/projects/1b12f8d03f4f4f4c8b5f37b804de18c5 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:33:36.384900 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70471e59-21da-461f-932b-54d86c4e7c86 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:36.385239 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70471e59-21da-461f-932b-54d86c4e7c86 None admin] PATH_INFO: `/v3/roles/member` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:36.389491 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3804cc67-0abc-45a3-828c-55657c7df00c None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:36.405436 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70471e59-21da-461f-932b-54d86c4e7c86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:33:36.410915 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-70471e59-21da-461f-932b-54d86c4e7c86 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 10:33:36.410915 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:36.410915 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:36.410915 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:36.410915 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:36.410915 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:36.410915 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:36.410915 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:36.410915 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:36.410915 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:36.410915 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:36.410915 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:36.410915 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:33:36.410915 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:33:36.410915 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:33:36.410915 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:33:36.411904 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:33:36.411904 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:33:36.411904 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:33:36.411904 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:33:36.411904 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:36.411904 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:36.411904 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:36.411904 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:36.411904 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:36.411904 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:36.411904 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:36.411904 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:36.411904 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:36.411904 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:36.411904 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:36.411904 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:36.412725 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:36.412725 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:36.412725 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:36.412725 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:36.412725 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:33:36.412725 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:33:36.412725 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:33:36.412725 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:33:36.412725 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:33:36.412725 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:33:36.412725 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 10:33:36.412725 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:33:36.412725 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 217/432] 10.222.0.15 () {56 vars in 1099 bytes} [Sun Sep 11 10:33:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:36.418861 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70471e59-21da-461f-932b-54d86c4e7c86 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:36.451956 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:36.521586 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3804cc67-0abc-45a3-828c-55657c7df00c None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:36.537741 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70471e59-21da-461f-932b-54d86c4e7c86 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:36.564546 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d82ea5-09df-467d-ab11-b856d4db27b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:36.565723 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d82ea5-09df-467d-ab11-b856d4db27b6 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:36.566306 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d82ea5-09df-467d-ab11-b856d4db27b6 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:36.566779 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d82ea5-09df-467d-ab11-b856d4db27b6 None admin] PATH_INFO: `/v3/groups/30c24407ddbb4d64af64c09b373e5a25` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:36.568147 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d82ea5-09df-467d-ab11-b856d4db27b6 None admin] RBAC: Authorizing `identity:get_group(group_id=30c24407ddbb4d64af64c09b373e5a25)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:36.569957 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d82ea5-09df-467d-ab11-b856d4db27b6 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:36.572492 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 216/433] 10.222.0.15 () {56 vars in 1180 bytes} [Sun Sep 11 10:33:36 2022] GET /identity/v3/groups/30c24407ddbb4d64af64c09b373e5a25 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:33:36.579569 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d82ea5-09df-467d-ab11-b856d4db27b6 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:36.580367 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21149c4-4b5e-4e0b-b173-d74ccd47a14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:36.581119 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21149c4-4b5e-4e0b-b173-d74ccd47a14c None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:36.581537 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21149c4-4b5e-4e0b-b173-d74ccd47a14c None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:36.581969 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21149c4-4b5e-4e0b-b173-d74ccd47a14c None admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:36.582762 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21149c4-4b5e-4e0b-b173-d74ccd47a14c None admin] RBAC: Adding query filter params (name=member) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:36.583203 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21149c4-4b5e-4e0b-b173-d74ccd47a14c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:36.584717 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21149c4-4b5e-4e0b-b173-d74ccd47a14c None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:36.597806 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 218/434] 10.222.0.15 () {56 vars in 1101 bytes} [Sun Sep 11 10:33:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:36.601205 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21149c4-4b5e-4e0b-b173-d74ccd47a14c None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:36.656763 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:36.707552 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d82ea5-09df-467d-ab11-b856d4db27b6 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:36.719755 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:36.746776 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21149c4-4b5e-4e0b-b173-d74ccd47a14c None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:36.761645 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06527b2e-9a68-41f6-beec-8a39751273a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:36.762729 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06527b2e-9a68-41f6-beec-8a39751273a5 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:36.763592 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06527b2e-9a68-41f6-beec-8a39751273a5 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:36.764300 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06527b2e-9a68-41f6-beec-8a39751273a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:36.765205 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06527b2e-9a68-41f6-beec-8a39751273a5 None admin] RBAC: Adding query filter params (group.id=30c24407ddbb4d64af64c09b373e5a25, role.id=cdb3aa1bd85b4694964629ff7cf8039b, scope.project.id=1b12f8d03f4f4f4c8b5f37b804de18c5) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:36.765733 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06527b2e-9a68-41f6-beec-8a39751273a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:36.767954 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06527b2e-9a68-41f6-beec-8a39751273a5 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:36.788009 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 217/435] 10.222.0.15 () {56 vars in 1376 bytes} [Sun Sep 11 10:33:36 2022] GET /identity/v3/role_assignments?group.id=30c24407ddbb4d64af64c09b373e5a25&scope.project.id=1b12f8d03f4f4f4c8b5f37b804de18c5&role.id=cdb3aa1bd85b4694964629ff7cf8039b => generated 611 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:36.793729 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3941ba-a216-4a97-9039-781811524d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:36.794360 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3941ba-a216-4a97-9039-781811524d00 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:36.794638 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3941ba-a216-4a97-9039-781811524d00 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:36.794904 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3941ba-a216-4a97-9039-781811524d00 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:36.831523 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681cfe75-9414-4f2b-a729-58106efee261 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:36.831885 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681cfe75-9414-4f2b-a729-58106efee261 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:36.832117 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681cfe75-9414-4f2b-a729-58106efee261 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:36.833624 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 218/436] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:33:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 10:33:36.839252 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4613531b-c6ec-42b2-b285-c317760f338d None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:36.839576 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4613531b-c6ec-42b2-b285-c317760f338d None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:36.839859 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4613531b-c6ec-42b2-b285-c317760f338d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:36.860343 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-4613531b-c6ec-42b2-b285-c317760f338d None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:33:36.861652 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-4613531b-c6ec-42b2-b285-c317760f338d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:33:36.875829 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3941ba-a216-4a97-9039-781811524d00 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:36.877306 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3941ba-a216-4a97-9039-781811524d00 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:36.942800 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 219/437] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:33:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 10:33:36.944637 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c0b104-bc21-4f69-995a-c5555b8592dc None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:36.945056 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c0b104-bc21-4f69-995a-c5555b8592dc None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:36.946094 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c0b104-bc21-4f69-995a-c5555b8592dc None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:36.951761 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 220/438] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:33:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:33:36.953225 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e486ac86-8b5c-48c6-a75a-69947b3b61e5 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:36.953508 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e486ac86-8b5c-48c6-a75a-69947b3b61e5 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:36.954353 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e486ac86-8b5c-48c6-a75a-69947b3b61e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:36.978645 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-e486ac86-8b5c-48c6-a75a-69947b3b61e5 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:33:36.979916 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-e486ac86-8b5c-48c6-a75a-69947b3b61e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:33:36.988654 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:37.019001 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 219/439] 10.222.0.15 () {58 vars in 1282 bytes} [Sun Sep 11 10:33:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 10:33:37.025837 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3941ba-a216-4a97-9039-781811524d00 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:37.051453 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:37.083932 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 221/440] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:33:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:33:37.137033 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3941ba-a216-4a97-9039-781811524d00 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:37.173125 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c7ea4b-89b3-4977-9c56-467f7771ace1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:37.173759 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c7ea4b-89b3-4977-9c56-467f7771ace1 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:37.174038 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c7ea4b-89b3-4977-9c56-467f7771ace1 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:37.174309 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c7ea4b-89b3-4977-9c56-467f7771ace1 None admin] PATH_INFO: `/v3/projects/6d140898a8684238bddc9872a32eb94c` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:37.175524 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c7ea4b-89b3-4977-9c56-467f7771ace1 None admin] RBAC: Authorizing `identity:get_project(project_id=6d140898a8684238bddc9872a32eb94c)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:37.176957 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c7ea4b-89b3-4977-9c56-467f7771ace1 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:37.178589 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 220/441] 10.222.0.15 () {56 vars in 1186 bytes} [Sun Sep 11 10:33:37 2022] GET /identity/v3/projects/6d140898a8684238bddc9872a32eb94c => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:33:37.184703 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e486ac86-8b5c-48c6-a75a-69947b3b61e5 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:37.224119 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c7ea4b-89b3-4977-9c56-467f7771ace1 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:37.293770 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:37.312955 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e486ac86-8b5c-48c6-a75a-69947b3b61e5 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:37.352588 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c7ea4b-89b3-4977-9c56-467f7771ace1 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:37.354059 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38da40c9-1e5a-4f6c-8f54-8cf7517156c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:37.355502 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da40c9-1e5a-4f6c-8f54-8cf7517156c0 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:37.355805 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da40c9-1e5a-4f6c-8f54-8cf7517156c0 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:37.356080 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da40c9-1e5a-4f6c-8f54-8cf7517156c0 None admin] PATH_INFO: `/v3/groups/30c24407ddbb4d64af64c09b373e5a25` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:37.357270 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38da40c9-1e5a-4f6c-8f54-8cf7517156c0 None admin] RBAC: Authorizing `identity:get_group(group_id=30c24407ddbb4d64af64c09b373e5a25)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:37.358703 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38da40c9-1e5a-4f6c-8f54-8cf7517156c0 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:37.360862 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 222/442] 10.222.0.15 () {56 vars in 1180 bytes} [Sun Sep 11 10:33:37 2022] GET /identity/v3/groups/30c24407ddbb4d64af64c09b373e5a25 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:37.366385 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38da40c9-1e5a-4f6c-8f54-8cf7517156c0 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:37.390961 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af86b196-eb0a-467c-8a5e-a9e671c05f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:37.391627 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af86b196-eb0a-467c-8a5e-a9e671c05f4e None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:37.391913 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af86b196-eb0a-467c-8a5e-a9e671c05f4e None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:37.392187 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af86b196-eb0a-467c-8a5e-a9e671c05f4e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:37.402788 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af86b196-eb0a-467c-8a5e-a9e671c05f4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:33:37.408028 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-af86b196-eb0a-467c-8a5e-a9e671c05f4e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 10:33:37.408028 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:37.408028 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:37.408028 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:37.408028 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:37.408028 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:37.408028 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:37.408028 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:37.408028 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:37.408028 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:37.408028 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:37.408028 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:37.408028 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:33:37.408028 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:33:37.408028 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:33:37.408028 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:33:37.408919 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:33:37.408919 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:33:37.408919 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:33:37.408919 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:33:37.408919 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:37.408919 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:37.408919 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:37.408919 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:37.408919 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:37.408919 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:37.408919 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:37.408919 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:37.408919 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:37.408919 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:37.408919 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:37.408919 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:37.409750 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:37.409750 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:37.409750 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:37.409750 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:37.409750 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:33:37.409750 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:33:37.409750 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:33:37.409750 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:33:37.409750 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:33:37.409750 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:33:37.409750 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 10:33:37.409750 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:33:37.409750 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 221/443] 10.222.0.15 () {56 vars in 1096 bytes} [Sun Sep 11 10:33:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 10:33:37.410421 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af86b196-eb0a-467c-8a5e-a9e671c05f4e None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:37.419749 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:37.470305 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38da40c9-1e5a-4f6c-8f54-8cf7517156c0 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:37.505106 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd19c88-3b69-436b-b44e-6e3086a638df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:37.505632 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd19c88-3b69-436b-b44e-6e3086a638df None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:37.505915 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd19c88-3b69-436b-b44e-6e3086a638df None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:37.506188 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd19c88-3b69-436b-b44e-6e3086a638df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:37.506903 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd19c88-3b69-436b-b44e-6e3086a638df None admin] RBAC: Adding query filter params (group.id=30c24407ddbb4d64af64c09b373e5a25, role.id=f836487a66c74c05838133c9d0614bbf, scope.project.id=6d140898a8684238bddc9872a32eb94c) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:37.507207 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd19c88-3b69-436b-b44e-6e3086a638df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:37.508695 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd19c88-3b69-436b-b44e-6e3086a638df None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:37.509787 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af86b196-eb0a-467c-8a5e-a9e671c05f4e None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:37.514489 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 223/444] 10.222.0.15 () {56 vars in 1376 bytes} [Sun Sep 11 10:33:37 2022] GET /identity/v3/role_assignments?group.id=30c24407ddbb4d64af64c09b373e5a25&scope.project.id=6d140898a8684238bddc9872a32eb94c&role.id=f836487a66c74c05838133c9d0614bbf => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:33:37.515275 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd19c88-3b69-436b-b44e-6e3086a638df None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:37.530885 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:37.557335 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b229f40a-c9fa-46c9-92ad-78b0c9c5c2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:37.558205 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b229f40a-c9fa-46c9-92ad-78b0c9c5c2a6 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:37.558370 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b229f40a-c9fa-46c9-92ad-78b0c9c5c2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:37.558714 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b229f40a-c9fa-46c9-92ad-78b0c9c5c2a6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:37.570459 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b229f40a-c9fa-46c9-92ad-78b0c9c5c2a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:33:37.576033 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-b229f40a-c9fa-46c9-92ad-78b0c9c5c2a6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 10:33:37.576033 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:37.576033 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:37.576033 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:37.576033 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:37.576033 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:37.576033 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:37.576033 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:37.576033 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:37.576033 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:37.576033 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:37.576033 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:37.576033 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:33:37.576033 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:33:37.576033 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:33:37.576033 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:33:37.577040 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:33:37.577040 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:33:37.577040 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:33:37.577040 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:33:37.577040 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:37.577040 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:37.577040 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:37.577040 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:37.577040 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:37.577040 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:37.577040 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:37.577040 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:37.577040 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:37.577040 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:37.577040 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:37.577040 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:37.577840 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:37.577840 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:37.577840 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:37.577840 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:37.577840 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:33:37.577840 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:33:37.577840 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:33:37.577840 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:33:37.577840 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:33:37.577840 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:33:37.577840 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 10:33:37.577840 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:33:37.577840 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 222/445] 10.222.0.15 () {56 vars in 1114 bytes} [Sun Sep 11 10:33:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:37.583144 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b229f40a-c9fa-46c9-92ad-78b0c9c5c2a6 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:37.644272 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd19c88-3b69-436b-b44e-6e3086a638df None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:37.662299 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:37.686628 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2ac6f7-90af-402a-a288-4783853ea0fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:37.687252 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2ac6f7-90af-402a-a288-4783853ea0fe None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:37.687559 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2ac6f7-90af-402a-a288-4783853ea0fe None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:37.687833 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2ac6f7-90af-402a-a288-4783853ea0fe None admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:37.688551 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2ac6f7-90af-402a-a288-4783853ea0fe None admin] RBAC: Adding query filter params (name=admin) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:37.688854 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2ac6f7-90af-402a-a288-4783853ea0fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:37.690241 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2ac6f7-90af-402a-a288-4783853ea0fe None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:37.694861 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b229f40a-c9fa-46c9-92ad-78b0c9c5c2a6 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:37.701423 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 224/446] 10.222.0.15 () {56 vars in 1099 bytes} [Sun Sep 11 10:33:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:37.707887 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2ac6f7-90af-402a-a288-4783853ea0fe None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:37.738002 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aa8dd9-faf5-402c-bad3-6524c65a5d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:37.738666 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa8dd9-faf5-402c-bad3-6524c65a5d27 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:37.738950 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa8dd9-faf5-402c-bad3-6524c65a5d27 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:37.739223 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa8dd9-faf5-402c-bad3-6524c65a5d27 None admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:37.739946 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aa8dd9-faf5-402c-bad3-6524c65a5d27 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:37.740250 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aa8dd9-faf5-402c-bad3-6524c65a5d27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:37.741639 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aa8dd9-faf5-402c-bad3-6524c65a5d27 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:37.749959 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 223/447] 10.222.0.15 () {56 vars in 1111 bytes} [Sun Sep 11 10:33:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:33:37.755676 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aa8dd9-faf5-402c-bad3-6524c65a5d27 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:37.794410 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:37.829811 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2ac6f7-90af-402a-a288-4783853ea0fe None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:37.872747 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431c2451-ae20-4254-aa18-4df930e2e82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:37.873024 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431c2451-ae20-4254-aa18-4df930e2e82d None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:37.873282 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431c2451-ae20-4254-aa18-4df930e2e82d None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:37.873487 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aa8dd9-faf5-402c-bad3-6524c65a5d27 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:37.873602 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431c2451-ae20-4254-aa18-4df930e2e82d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:37.912973 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0a3024-43d3-4fab-b7b0-0795a66acec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:37.913588 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0a3024-43d3-4fab-b7b0-0795a66acec7 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:37.914573 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0a3024-43d3-4fab-b7b0-0795a66acec7 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:37.914849 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0a3024-43d3-4fab-b7b0-0795a66acec7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:37.926692 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:37.951107 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431c2451-ae20-4254-aa18-4df930e2e82d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:37.952591 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431c2451-ae20-4254-aa18-4df930e2e82d None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:37.983161 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0a3024-43d3-4fab-b7b0-0795a66acec7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:37.984628 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0a3024-43d3-4fab-b7b0-0795a66acec7 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:38.017749 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:38.051746 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 225/448] 10.222.0.15 () {58 vars in 1282 bytes} [Sun Sep 11 10:33:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 10:33:38.057949 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431c2451-ae20-4254-aa18-4df930e2e82d None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:38.086513 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 224/449] 10.222.0.15 () {58 vars in 1282 bytes} [Sun Sep 11 10:33:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 10:33:38.092014 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0a3024-43d3-4fab-b7b0-0795a66acec7 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:38.133998 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:38.157676 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431c2451-ae20-4254-aa18-4df930e2e82d None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:38.189398 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0a3024-43d3-4fab-b7b0-0795a66acec7 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:38.191243 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ac3728-a6da-45ae-aea9-5a787eaf7880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:38.191898 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac3728-a6da-45ae-aea9-5a787eaf7880 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:38.192170 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac3728-a6da-45ae-aea9-5a787eaf7880 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:38.192443 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac3728-a6da-45ae-aea9-5a787eaf7880 None admin] PATH_INFO: `/v3/projects/dcf7c8cb1c844da09226c671da5a163e` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:38.193536 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ac3728-a6da-45ae-aea9-5a787eaf7880 None admin] RBAC: Authorizing `identity:get_project(project_id=dcf7c8cb1c844da09226c671da5a163e)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:38.195526 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ac3728-a6da-45ae-aea9-5a787eaf7880 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:38.197794 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 226/450] 10.222.0.15 () {56 vars in 1186 bytes} [Sun Sep 11 10:33:38 2022] GET /identity/v3/projects/dcf7c8cb1c844da09226c671da5a163e => generated 344 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:33:38.203043 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ac3728-a6da-45ae-aea9-5a787eaf7880 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:38.223161 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414cd704-5c89-4d65-8171-902d91c0ba6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:38.223819 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414cd704-5c89-4d65-8171-902d91c0ba6f None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:38.224095 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414cd704-5c89-4d65-8171-902d91c0ba6f None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:38.224363 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414cd704-5c89-4d65-8171-902d91c0ba6f None admin] PATH_INFO: `/v3/projects/6d140898a8684238bddc9872a32eb94c` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:38.225460 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414cd704-5c89-4d65-8171-902d91c0ba6f None admin] RBAC: Authorizing `identity:get_project(project_id=6d140898a8684238bddc9872a32eb94c)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:38.228012 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414cd704-5c89-4d65-8171-902d91c0ba6f None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:38.234048 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 225/451] 10.222.0.15 () {56 vars in 1186 bytes} [Sun Sep 11 10:33:38 2022] GET /identity/v3/projects/6d140898a8684238bddc9872a32eb94c => generated 302 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:33:38.235018 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414cd704-5c89-4d65-8171-902d91c0ba6f None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:38.237551 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:38.306191 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ac3728-a6da-45ae-aea9-5a787eaf7880 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:38.338685 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414cd704-5c89-4d65-8171-902d91c0ba6f None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:38.340819 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b4d0b8-9af7-459b-b23b-cb1be50f5aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:38.341447 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b4d0b8-9af7-459b-b23b-cb1be50f5aaf None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:38.341725 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b4d0b8-9af7-459b-b23b-cb1be50f5aaf None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:38.341996 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b4d0b8-9af7-459b-b23b-cb1be50f5aaf None admin] PATH_INFO: `/v3/groups/dfe4e0ab839a4157aa444073fea888a2` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:38.343100 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b4d0b8-9af7-459b-b23b-cb1be50f5aaf None admin] RBAC: Authorizing `identity:get_group(group_id=dfe4e0ab839a4157aa444073fea888a2)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:38.344589 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b4d0b8-9af7-459b-b23b-cb1be50f5aaf None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:38.346139 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 227/452] 10.222.0.15 () {56 vars in 1180 bytes} [Sun Sep 11 10:33:38 2022] GET /identity/v3/groups/dfe4e0ab839a4157aa444073fea888a2 => generated 229 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:33:38.352595 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b4d0b8-9af7-459b-b23b-cb1be50f5aaf None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:38.366618 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:38.374023 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a111b6f1-40ca-445f-8fc0-015dd3d045da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:38.374653 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a111b6f1-40ca-445f-8fc0-015dd3d045da None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:38.375366 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a111b6f1-40ca-445f-8fc0-015dd3d045da None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:38.375678 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a111b6f1-40ca-445f-8fc0-015dd3d045da None admin] PATH_INFO: `/v3/groups/30c24407ddbb4d64af64c09b373e5a25` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:38.376793 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a111b6f1-40ca-445f-8fc0-015dd3d045da None admin] RBAC: Authorizing `identity:get_group(group_id=30c24407ddbb4d64af64c09b373e5a25)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:38.378536 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a111b6f1-40ca-445f-8fc0-015dd3d045da None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:38.380138 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 226/453] 10.222.0.15 () {56 vars in 1180 bytes} [Sun Sep 11 10:33:38 2022] GET /identity/v3/groups/30c24407ddbb4d64af64c09b373e5a25 => generated 226 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:38.385805 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a111b6f1-40ca-445f-8fc0-015dd3d045da None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:38.452569 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:38.453688 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b4d0b8-9af7-459b-b23b-cb1be50f5aaf None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:38.483326 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a111b6f1-40ca-445f-8fc0-015dd3d045da None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:38.487574 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abc34e2-cb5f-4481-980e-96276d9f1377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:38.488181 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abc34e2-cb5f-4481-980e-96276d9f1377 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:38.488466 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abc34e2-cb5f-4481-980e-96276d9f1377 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:38.488740 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abc34e2-cb5f-4481-980e-96276d9f1377 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:38.489459 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abc34e2-cb5f-4481-980e-96276d9f1377 None admin] RBAC: Adding query filter params (group.id=dfe4e0ab839a4157aa444073fea888a2, role.id=2cb7d73f0ea8414296367573f6902340, scope.project.id=dcf7c8cb1c844da09226c671da5a163e) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:38.489768 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abc34e2-cb5f-4481-980e-96276d9f1377 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:38.491197 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abc34e2-cb5f-4481-980e-96276d9f1377 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:38.497387 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 228/454] 10.222.0.15 () {56 vars in 1376 bytes} [Sun Sep 11 10:33:38 2022] GET /identity/v3/role_assignments?group.id=dfe4e0ab839a4157aa444073fea888a2&scope.project.id=dcf7c8cb1c844da09226c671da5a163e&role.id=2cb7d73f0ea8414296367573f6902340 => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:33:38.517530 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6b0a61-d382-4686-812e-31f7067d893a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:38.518155 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6b0a61-d382-4686-812e-31f7067d893a None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:38.518435 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6b0a61-d382-4686-812e-31f7067d893a None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:38.518707 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6b0a61-d382-4686-812e-31f7067d893a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:38.519483 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6b0a61-d382-4686-812e-31f7067d893a None admin] RBAC: Adding query filter params (group.id=30c24407ddbb4d64af64c09b373e5a25, role.id=cdb3aa1bd85b4694964629ff7cf8039b, scope.project.id=6d140898a8684238bddc9872a32eb94c) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:38.519793 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6b0a61-d382-4686-812e-31f7067d893a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:38.521262 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6b0a61-d382-4686-812e-31f7067d893a None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:38.528709 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 227/455] 10.222.0.15 () {56 vars in 1376 bytes} [Sun Sep 11 10:33:38 2022] GET /identity/v3/role_assignments?group.id=30c24407ddbb4d64af64c09b373e5a25&scope.project.id=6d140898a8684238bddc9872a32eb94c&role.id=cdb3aa1bd85b4694964629ff7cf8039b => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:33:41.427473 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d17e093-fc22-4318-b1bc-12b989843fd8 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:41.427985 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d17e093-fc22-4318-b1bc-12b989843fd8 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:41.428059 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d17e093-fc22-4318-b1bc-12b989843fd8 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:41.429883 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 229/456] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:33:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:33:41.435001 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad4ec0-23f3-486d-9ac0-936a7c42ed70 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:41.435411 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad4ec0-23f3-486d-9ac0-936a7c42ed70 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:41.435626 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad4ec0-23f3-486d-9ac0-936a7c42ed70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:41.457899 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-7aad4ec0-23f3-486d-9ac0-936a7c42ed70 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:33:41.459086 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-7aad4ec0-23f3-486d-9ac0-936a7c42ed70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:33:41.553989 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 228/457] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:33:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:33:41.701496 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294b1974-ec0b-4b29-a342-2e3a6dd36f46 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:41.701788 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294b1974-ec0b-4b29-a342-2e3a6dd36f46 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:41.701954 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294b1974-ec0b-4b29-a342-2e3a6dd36f46 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:41.703564 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 230/458] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:33:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:33:41.708222 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e777009-1b26-4ad8-9422-4e7cc87ad501 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:41.708454 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f995db95-f5e9-4293-9edd-27658ee0c708 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:41.708559 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e777009-1b26-4ad8-9422-4e7cc87ad501 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:41.708749 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f995db95-f5e9-4293-9edd-27658ee0c708 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:41.708845 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e777009-1b26-4ad8-9422-4e7cc87ad501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:41.709025 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f995db95-f5e9-4293-9edd-27658ee0c708 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:41.711112 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 229/459] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:33:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:33:41.719045 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df658e11-cc84-4f28-9443-4ebc351c0b8c None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:41.719496 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df658e11-cc84-4f28-9443-4ebc351c0b8c None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:41.719669 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df658e11-cc84-4f28-9443-4ebc351c0b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:41.733028 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-6e777009-1b26-4ad8-9422-4e7cc87ad501 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:33:41.734606 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-6e777009-1b26-4ad8-9422-4e7cc87ad501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:33:41.756977 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-df658e11-cc84-4f28-9443-4ebc351c0b8c None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:33:41.758083 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-df658e11-cc84-4f28-9443-4ebc351c0b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:33:41.834411 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:41.836974 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 231/460] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:33:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:33:41.845272 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 230/461] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:33:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:33:41.971658 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e777009-1b26-4ad8-9422-4e7cc87ad501 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:41.982152 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:42.008391 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89867ce4-768c-47ff-ac42-d51bb2d2d9f1 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:42.008815 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89867ce4-768c-47ff-ac42-d51bb2d2d9f1 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:42.009004 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89867ce4-768c-47ff-ac42-d51bb2d2d9f1 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:42.010565 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 231/462] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:33:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:33:42.015711 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fbb3fa-ffea-46b5-b72b-0fbcc6e2b3b8 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:42.015944 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fbb3fa-ffea-46b5-b72b-0fbcc6e2b3b8 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:42.016218 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fbb3fa-ffea-46b5-b72b-0fbcc6e2b3b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:42.049748 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-a2fbb3fa-ffea-46b5-b72b-0fbcc6e2b3b8 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:33:42.050938 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-a2fbb3fa-ffea-46b5-b72b-0fbcc6e2b3b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:33:42.098234 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e777009-1b26-4ad8-9422-4e7cc87ad501 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:42.150819 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b186df-c7c1-4863-9c84-e976716a8bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:42.151882 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b186df-c7c1-4863-9c84-e976716a8bd7 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:42.152352 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b186df-c7c1-4863-9c84-e976716a8bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:42.152834 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b186df-c7c1-4863-9c84-e976716a8bd7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:42.163244 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b186df-c7c1-4863-9c84-e976716a8bd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:42.167114 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b186df-c7c1-4863-9c84-e976716a8bd7 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:42.185039 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-79b186df-c7c1-4863-9c84-e976716a8bd7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:33:42.185039 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:42.185039 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:42.185039 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:42.185039 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:42.185039 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:42.185039 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:42.185039 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:42.185039 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:42.185039 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:42.185039 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:42.185039 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:42.185039 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:33:42.185039 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:33:42.185039 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:33:42.185039 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:33:42.186002 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:42.186002 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:42.186002 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:42.186002 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:42.186002 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:42.186002 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:42.186002 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:42.186002 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:42.186002 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:42.186002 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:42.186002 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:42.186002 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:42.186002 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:42.186002 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:42.186002 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:42.186002 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:42.186002 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:33:42.186865 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:33:42.186865 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:33:42.186865 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:33:42.187036 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 232/463] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:33:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 10:33:42.187107 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 232/464] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:33:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:42.187172 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fbb3fa-ffea-46b5-b72b-0fbcc6e2b3b8 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:42.187238 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b186df-c7c1-4863-9c84-e976716a8bd7 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:42.255540 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:42.310387 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b186df-c7c1-4863-9c84-e976716a8bd7 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:42.332509 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fbb3fa-ffea-46b5-b72b-0fbcc6e2b3b8 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:42.352507 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb08f77-09af-4e4d-8ca7-5d4fc8baf35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:42.353486 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb08f77-09af-4e4d-8ca7-5d4fc8baf35d None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:42.353962 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb08f77-09af-4e4d-8ca7-5d4fc8baf35d None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:42.354426 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb08f77-09af-4e4d-8ca7-5d4fc8baf35d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:42.364301 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:42.366202 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb08f77-09af-4e4d-8ca7-5d4fc8baf35d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:42.368931 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb08f77-09af-4e4d-8ca7-5d4fc8baf35d None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:42.377373 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ee4b90-3099-44e4-be55-0cdcd27319c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:42.378542 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ee4b90-3099-44e4-be55-0cdcd27319c0 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:42.378832 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ee4b90-3099-44e4-be55-0cdcd27319c0 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:42.379112 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ee4b90-3099-44e4-be55-0cdcd27319c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:42.385249 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-adb08f77-09af-4e4d-8ca7-5d4fc8baf35d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:33:42.385249 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:42.385249 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:42.385249 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:42.385249 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:42.385249 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:42.385249 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:42.385249 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:42.385249 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:42.385249 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:42.385249 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:42.385249 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:42.385249 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:33:42.385249 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:33:42.385249 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:33:42.385249 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:33:42.386137 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:42.386137 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:42.386137 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:42.386137 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:42.386137 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:42.386137 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:42.386137 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:42.386137 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:42.386137 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:42.386137 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:42.386137 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:42.386137 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:42.386137 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:42.386137 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:42.386137 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:42.386137 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:42.386137 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:33:42.387095 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:33:42.387095 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:33:42.387095 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:33:42.389231 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 233/465] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:33:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:42.390001 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ee4b90-3099-44e4-be55-0cdcd27319c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:42.390486 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb08f77-09af-4e4d-8ca7-5d4fc8baf35d None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:42.391498 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ee4b90-3099-44e4-be55-0cdcd27319c0 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:42.409157 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-80ee4b90-3099-44e4-be55-0cdcd27319c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:33:42.409157 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:42.409157 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:42.409157 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:42.409157 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:42.409157 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:42.409157 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:42.409157 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:42.409157 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:42.409157 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:42.409157 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:42.409157 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:42.409157 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:33:42.409157 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:33:42.409157 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:33:42.409157 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:33:42.410122 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:42.410122 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:42.410122 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:42.410122 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:42.410122 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:42.410122 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:42.410122 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:42.410122 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:42.410122 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:42.410122 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:42.410122 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:42.410122 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:42.410122 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:42.410122 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:42.410122 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:42.410122 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:42.410122 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:33:42.410974 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:33:42.410974 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:33:42.410974 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:33:42.418187 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 233/466] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:33:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:42.419382 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ee4b90-3099-44e4-be55-0cdcd27319c0 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:42.515908 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb08f77-09af-4e4d-8ca7-5d4fc8baf35d None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:42.527468 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:42.533733 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ee4b90-3099-44e4-be55-0cdcd27319c0 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:42.553358 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdd004b-5b40-4dd5-bde0-f60d8fa8772d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:42.553961 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdd004b-5b40-4dd5-bde0-f60d8fa8772d None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:42.554258 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdd004b-5b40-4dd5-bde0-f60d8fa8772d None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:42.554548 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdd004b-5b40-4dd5-bde0-f60d8fa8772d None admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:42.555604 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdd004b-5b40-4dd5-bde0-f60d8fa8772d None admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:42.555909 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdd004b-5b40-4dd5-bde0-f60d8fa8772d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:42.557287 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdd004b-5b40-4dd5-bde0-f60d8fa8772d None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:42.574098 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 234/467] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:33:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:42.574098 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdd004b-5b40-4dd5-bde0-f60d8fa8772d None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:42.574317 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c761d9-e9e5-442e-a85a-b71a1f6181d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:42.574317 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c761d9-e9e5-442e-a85a-b71a1f6181d6 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:42.574317 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c761d9-e9e5-442e-a85a-b71a1f6181d6 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:42.574317 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c761d9-e9e5-442e-a85a-b71a1f6181d6 None admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:42.574604 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c761d9-e9e5-442e-a85a-b71a1f6181d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:42.574604 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c761d9-e9e5-442e-a85a-b71a1f6181d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:42.575559 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c761d9-e9e5-442e-a85a-b71a1f6181d6 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:42.593468 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 234/468] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:33:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:42.594301 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c761d9-e9e5-442e-a85a-b71a1f6181d6 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:42.632601 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:42.686332 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdd004b-5b40-4dd5-bde0-f60d8fa8772d None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:42.703383 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c761d9-e9e5-442e-a85a-b71a1f6181d6 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:42.722731 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac26104-1308-4206-884a-c64569c9b987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:42.723646 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac26104-1308-4206-884a-c64569c9b987 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:42.723932 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac26104-1308-4206-884a-c64569c9b987 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:42.724204 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac26104-1308-4206-884a-c64569c9b987 None admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:42.724843 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac26104-1308-4206-884a-c64569c9b987 None admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:42.725145 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac26104-1308-4206-884a-c64569c9b987 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:42.727260 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac26104-1308-4206-884a-c64569c9b987 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:42.742572 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 235/469] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:33:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:42.742572 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac26104-1308-4206-884a-c64569c9b987 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:42.745171 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcbbd58-063c-4f3b-9ccc-2496dd34297c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:42.745891 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcbbd58-063c-4f3b-9ccc-2496dd34297c None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:42.746288 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcbbd58-063c-4f3b-9ccc-2496dd34297c None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:42.746559 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcbbd58-063c-4f3b-9ccc-2496dd34297c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:42.757098 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcbbd58-063c-4f3b-9ccc-2496dd34297c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:42.759786 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcbbd58-063c-4f3b-9ccc-2496dd34297c None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:42.774889 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:42.775778 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-ddcbbd58-063c-4f3b-9ccc-2496dd34297c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:33:42.775778 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:42.775778 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:42.775778 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:42.775778 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:42.775778 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:42.775778 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:42.775778 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:42.775778 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:42.775778 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:42.775778 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:42.775778 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:42.775778 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:33:42.775778 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:33:42.775778 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:33:42.775778 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:33:42.776982 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:42.776982 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:42.776982 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:42.776982 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:42.776982 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:42.776982 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:42.776982 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:42.776982 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:42.776982 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:42.776982 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:42.776982 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:42.776982 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:42.776982 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:42.776982 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:42.776982 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:42.776982 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:42.776982 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:33:42.777830 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:33:42.777830 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:33:42.777830 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:33:42.782115 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 235/470] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:33:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:42.782728 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcbbd58-063c-4f3b-9ccc-2496dd34297c None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:42.853143 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac26104-1308-4206-884a-c64569c9b987 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:42.874669 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:42.888741 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcbbd58-063c-4f3b-9ccc-2496dd34297c None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:42.889606 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9045aa19-2af1-48cd-9507-6069f1d14a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:42.890207 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9045aa19-2af1-48cd-9507-6069f1d14a1d None admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:42.890486 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9045aa19-2af1-48cd-9507-6069f1d14a1d None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:42.890751 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9045aa19-2af1-48cd-9507-6069f1d14a1d None admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:42.891386 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9045aa19-2af1-48cd-9507-6069f1d14a1d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:42.892844 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9045aa19-2af1-48cd-9507-6069f1d14a1d None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:42.917874 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 236/471] 10.222.0.15 () {60 vars in 1132 bytes} [Sun Sep 11 10:33:42 2022] POST /identity/v3/users => generated 248 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:33:42.918805 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9045aa19-2af1-48cd-9507-6069f1d14a1d None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:42.946007 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b507ef11-819e-45c8-a628-d1e19a9de77c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:42.948634 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b507ef11-819e-45c8-a628-d1e19a9de77c None admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:42.949439 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b507ef11-819e-45c8-a628-d1e19a9de77c None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:42.950115 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b507ef11-819e-45c8-a628-d1e19a9de77c None admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:42.952006 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b507ef11-819e-45c8-a628-d1e19a9de77c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:42.954505 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b507ef11-819e-45c8-a628-d1e19a9de77c None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:42.990648 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 236/472] 10.222.0.15 () {60 vars in 1132 bytes} [Sun Sep 11 10:33:42 2022] POST /identity/v3/users => generated 246 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:33:42.991959 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b507ef11-819e-45c8-a628-d1e19a9de77c None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:43.078068 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:43.105459 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9045aa19-2af1-48cd-9507-6069f1d14a1d None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:43.163622 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23cdfbb-1db8-40ce-8ca8-cc9e5f855b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:43.164048 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b507ef11-819e-45c8-a628-d1e19a9de77c None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:43.165071 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23cdfbb-1db8-40ce-8ca8-cc9e5f855b45 None admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:43.165563 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23cdfbb-1db8-40ce-8ca8-cc9e5f855b45 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:43.166071 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23cdfbb-1db8-40ce-8ca8-cc9e5f855b45 None admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:43.167472 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23cdfbb-1db8-40ce-8ca8-cc9e5f855b45 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:43.168336 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23cdfbb-1db8-40ce-8ca8-cc9e5f855b45 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:43.194382 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 237/473] 10.222.0.15 () {60 vars in 1132 bytes} [Sun Sep 11 10:33:42 2022] POST /identity/v3/users => generated 248 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:33:43.197073 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af889b-0cfd-414f-b85f-6d0b2c46bc7e None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:43.197371 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af889b-0cfd-414f-b85f-6d0b2c46bc7e None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:43.197645 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af889b-0cfd-414f-b85f-6d0b2c46bc7e None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:43.199717 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 238/474] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:33:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:33:43.205930 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a725f9-6138-4279-a7a5-00fa292f92e6 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:43.206218 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a725f9-6138-4279-a7a5-00fa292f92e6 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:43.206489 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a725f9-6138-4279-a7a5-00fa292f92e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:43.209899 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273a2568-9d27-4171-baa5-38cf6df8ff39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:43.210936 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a2568-9d27-4171-baa5-38cf6df8ff39 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:43.212169 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a2568-9d27-4171-baa5-38cf6df8ff39 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:43.212607 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a2568-9d27-4171-baa5-38cf6df8ff39 None admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:43.213501 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273a2568-9d27-4171-baa5-38cf6df8ff39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:43.213945 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273a2568-9d27-4171-baa5-38cf6df8ff39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:43.215518 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273a2568-9d27-4171-baa5-38cf6df8ff39 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:43.230098 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 237/475] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:33:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:43.231467 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-44a725f9-6138-4279-a7a5-00fa292f92e6 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:33:43.232097 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-44a725f9-6138-4279-a7a5-00fa292f92e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:33:43.237130 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273a2568-9d27-4171-baa5-38cf6df8ff39 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:43.243059 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:43.336094 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 239/476] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:33:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 10:33:43.381056 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273a2568-9d27-4171-baa5-38cf6df8ff39 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:43.419117 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5652fdd7-15c0-4f87-8c5b-29db22244274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:43.419986 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5652fdd7-15c0-4f87-8c5b-29db22244274 None admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:43.420403 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5652fdd7-15c0-4f87-8c5b-29db22244274 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:43.420814 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5652fdd7-15c0-4f87-8c5b-29db22244274 None admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:43.421595 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5652fdd7-15c0-4f87-8c5b-29db22244274 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:43.423246 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5652fdd7-15c0-4f87-8c5b-29db22244274 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:43.449834 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 238/477] 10.222.0.15 () {60 vars in 1132 bytes} [Sun Sep 11 10:33:43 2022] POST /identity/v3/users => generated 249 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 10:33:43.642647 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a725f9-6138-4279-a7a5-00fa292f92e6 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:43.722646 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:43.761746 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a725f9-6138-4279-a7a5-00fa292f92e6 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:43.798658 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748cb147-44b6-4945-9789-e11626cb6794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:43.799316 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748cb147-44b6-4945-9789-e11626cb6794 None admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:43.799639 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748cb147-44b6-4945-9789-e11626cb6794 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:43.799896 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748cb147-44b6-4945-9789-e11626cb6794 None admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:43.800560 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748cb147-44b6-4945-9789-e11626cb6794 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:43.801915 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748cb147-44b6-4945-9789-e11626cb6794 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:43.810985 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.sql.core [None req-748cb147-44b6-4945-9789-e11626cb6794 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 11 10:33:43.810985 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 11 10:33:43.810985 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [parameters: {'id': '389c7ce6f1214657bc9c24fd99d74e68', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 11 10:33:43.810985 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=90653) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 11 10:33:43.828986 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-748cb147-44b6-4945-9789-e11626cb6794 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 11 10:33:43.828986 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:43.828986 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 11 10:33:43.828986 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 11 10:33:43.828986 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 11 10:33:43.828986 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 11 10:33:43.828986 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 11 10:33:43.828986 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application result = self._query(query) Sep 11 10:33:43.828986 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 11 10:33:43.828986 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application conn.query(q) Sep 11 10:33:43.828986 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 11 10:33:43.828986 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 11 10:33:43.828986 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 11 10:33:43.828986 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application result.read() Sep 11 10:33:43.828986 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 11 10:33:43.829974 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 11 10:33:43.829974 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 11 10:33:43.829974 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 11 10:33:43.829974 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 11 10:33:43.829974 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 11 10:33:43.829974 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 11 10:33:43.829974 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 11 10:33:43.829974 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 11 10:33:43.829974 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:33:43.829974 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 11 10:33:43.829974 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:33:43.829974 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:43.829974 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 11 10:33:43.829974 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 11 10:33:43.829974 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 11 10:33:43.829974 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return ref.to_dict() Sep 11 10:33:43.829974 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 11 10:33:43.829974 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application next(self.gen) Sep 11 10:33:43.830873 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 11 10:33:43.830873 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application yield resource Sep 11 10:33:43.830873 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 11 10:33:43.830873 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application next(self.gen) Sep 11 10:33:43.830873 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 11 10:33:43.830873 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application self.session.rollback() Sep 11 10:33:43.830873 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 11 10:33:43.830873 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application self.force_reraise() Sep 11 10:33:43.830873 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 11 10:33:43.830873 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise self.value Sep 11 10:33:43.830873 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 11 10:33:43.830873 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 11 10:33:43.830873 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 11 10:33:43.830873 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application session.commit() Sep 11 10:33:43.830873 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 11 10:33:43.830873 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 11 10:33:43.830873 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 11 10:33:43.831789 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application self._prepare_impl() Sep 11 10:33:43.831789 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 11 10:33:43.831789 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application self.session.flush() Sep 11 10:33:43.831789 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 11 10:33:43.831789 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application self._flush(objects) Sep 11 10:33:43.831789 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 11 10:33:43.831789 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 11 10:33:43.831789 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 11 10:33:43.831789 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application compat.raise_( Sep 11 10:33:43.831789 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 11 10:33:43.831789 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception Sep 11 10:33:43.831789 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 11 10:33:43.831789 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application flush_context.execute() Sep 11 10:33:43.831789 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 11 10:33:43.831789 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rec.execute(self) Sep 11 10:33:43.831789 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 11 10:33:43.831789 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 11 10:33:43.833315 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 11 10:33:43.833315 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 11 10:33:43.833315 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 11 10:33:43.833315 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 11 10:33:43.833315 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 11 10:33:43.833315 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 11 10:33:43.833315 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 11 10:33:43.833315 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 11 10:33:43.833315 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 11 10:33:43.833315 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 11 10:33:43.833315 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 11 10:33:43.833315 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 11 10:33:43.833315 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 11 10:33:43.833315 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 11 10:33:43.833315 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 11 10:33:43.833315 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception Sep 11 10:33:43.834129 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 11 10:33:43.834129 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 11 10:33:43.834129 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 11 10:33:43.834129 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 11 10:33:43.834129 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 11 10:33:43.834129 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application result = self._query(query) Sep 11 10:33:43.834129 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 11 10:33:43.834129 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application conn.query(q) Sep 11 10:33:43.834129 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 11 10:33:43.834129 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 11 10:33:43.834129 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 11 10:33:43.834129 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application result.read() Sep 11 10:33:43.834129 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 11 10:33:43.834129 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 11 10:33:43.834129 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 11 10:33:43.834129 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 11 10:33:43.834918 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 11 10:33:43.834918 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 11 10:33:43.834918 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 11 10:33:43.834918 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 11 10:33:43.834918 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 11 10:33:43.834918 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 11 10:33:43.834918 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [parameters: {'id': '389c7ce6f1214657bc9c24fd99d74e68', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 11 10:33:43.834918 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 11 10:33:43.834918 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:33:43.834918 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 10:33:43.834918 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:33:43.834918 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:43.834918 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:43.834918 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:43.834918 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:43.834918 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:43.837709 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:43.837709 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:43.837709 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:43.837709 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:43.837709 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:43.837709 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:43.837709 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 11 10:33:43.837709 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 11 10:33:43.837709 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:43.837709 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:43.837709 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 11 10:33:43.837709 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 11 10:33:43.837709 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 11 10:33:43.837709 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 11 10:33:43.837709 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 11 10:33:43.837709 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:33:43.839781 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 240/478] 10.222.0.15 () {60 vars in 1131 bytes} [Sun Sep 11 10:33:43 2022] POST /identity/v3/roles => generated 144 bytes in 188 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:43.839914 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5652fdd7-15c0-4f87-8c5b-29db22244274 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:43.934796 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:43.989489 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5652fdd7-15c0-4f87-8c5b-29db22244274 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:44.025265 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bc5f65-8a3e-4e6a-9a65-61511f2c2cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:44.025908 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bc5f65-8a3e-4e6a-9a65-61511f2c2cb5 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:44.026188 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bc5f65-8a3e-4e6a-9a65-61511f2c2cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:44.026461 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bc5f65-8a3e-4e6a-9a65-61511f2c2cb5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:44.037040 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bc5f65-8a3e-4e6a-9a65-61511f2c2cb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:33:44.042423 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-c9bc5f65-8a3e-4e6a-9a65-61511f2c2cb5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 10:33:44.042423 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:44.042423 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:44.042423 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:44.042423 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:44.042423 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:44.042423 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:44.042423 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:44.042423 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:44.042423 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:44.042423 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:44.042423 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:44.042423 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:33:44.042423 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:33:44.042423 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:33:44.042423 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:33:44.043545 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:33:44.043545 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:33:44.043545 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:33:44.043545 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:33:44.043545 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:44.043545 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:44.043545 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:44.043545 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:44.043545 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:44.043545 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:44.043545 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:44.043545 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:44.043545 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:44.043545 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:44.043545 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:44.043545 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:44.044459 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:44.044459 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:44.044459 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:44.044459 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:44.044459 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:33:44.044459 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:33:44.044459 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:33:44.044459 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:33:44.044459 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:33:44.044459 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:33:44.044459 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 10:33:44.044459 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:33:44.044459 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 239/479] 10.222.0.15 () {56 vars in 1114 bytes} [Sun Sep 11 10:33:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:44.051114 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748cb147-44b6-4945-9789-e11626cb6794 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:44.156037 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748cb147-44b6-4945-9789-e11626cb6794 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:44.191521 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb2ab0f-2e1e-4095-80b9-66fc11fca43a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:44.192191 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb2ab0f-2e1e-4095-80b9-66fc11fca43a None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:44.192473 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb2ab0f-2e1e-4095-80b9-66fc11fca43a None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:44.192744 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb2ab0f-2e1e-4095-80b9-66fc11fca43a None admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:44.193426 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb2ab0f-2e1e-4095-80b9-66fc11fca43a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:44.193728 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb2ab0f-2e1e-4095-80b9-66fc11fca43a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:44.195699 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb2ab0f-2e1e-4095-80b9-66fc11fca43a None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:44.198422 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:44.205537 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 241/480] 10.222.0.15 () {56 vars in 1111 bytes} [Sun Sep 11 10:33:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:45.706751 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ca320-5fe2-4abc-a2ff-66b2433136b4 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:45.707679 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ca320-5fe2-4abc-a2ff-66b2433136b4 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:45.708131 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5ca320-5fe2-4abc-a2ff-66b2433136b4 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:45.709916 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 240/481] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:33:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:33:45.714939 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacf3f2f-0e0d-4a9d-8095-3b1633a35773 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:45.715722 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacf3f2f-0e0d-4a9d-8095-3b1633a35773 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:45.716190 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacf3f2f-0e0d-4a9d-8095-3b1633a35773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:45.745008 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-aacf3f2f-0e0d-4a9d-8095-3b1633a35773 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:33:45.746479 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-aacf3f2f-0e0d-4a9d-8095-3b1633a35773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:33:45.823011 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 242/482] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:33:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:33:46.024189 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7181e58-d833-4b09-abc8-9601eaf05309 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:46.024467 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7181e58-d833-4b09-abc8-9601eaf05309 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:46.024745 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7181e58-d833-4b09-abc8-9601eaf05309 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:46.026347 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 241/483] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:33:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:33:46.031652 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91916e5a-b0cd-4ee4-b948-5df438128954 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:46.031941 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91916e5a-b0cd-4ee4-b948-5df438128954 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:46.032219 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91916e5a-b0cd-4ee4-b948-5df438128954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:46.054100 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-91916e5a-b0cd-4ee4-b948-5df438128954 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:33:46.055870 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-91916e5a-b0cd-4ee4-b948-5df438128954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:33:46.105865 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49958104-2ca0-4c23-be58-9f861dcc2de4 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:46.106145 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49958104-2ca0-4c23-be58-9f861dcc2de4 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:46.106416 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49958104-2ca0-4c23-be58-9f861dcc2de4 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:46.112092 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 242/484] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:33:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:33:46.113462 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e2b0f8-0d6f-40dd-939e-58ba4a77cdc8 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:46.113923 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e2b0f8-0d6f-40dd-939e-58ba4a77cdc8 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:46.114410 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e2b0f8-0d6f-40dd-939e-58ba4a77cdc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:46.130592 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 243/485] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:33:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:33:46.152585 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-65e2b0f8-0d6f-40dd-939e-58ba4a77cdc8 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:33:46.154082 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-65e2b0f8-0d6f-40dd-939e-58ba4a77cdc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:33:46.198888 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91916e5a-b0cd-4ee4-b948-5df438128954 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:46.246317 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 243/486] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:33:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:33:46.307535 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:46.322898 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b48351-c4c4-4e82-bbb8-f2223c9ef518 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:46.323381 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b48351-c4c4-4e82-bbb8-f2223c9ef518 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:46.323894 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b48351-c4c4-4e82-bbb8-f2223c9ef518 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:46.325694 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 244/487] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:33:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 10:33:46.340627 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d665ab22-86aa-4973-bce7-1ce3c610fd7e None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:46.341174 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d665ab22-86aa-4973-bce7-1ce3c610fd7e None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:46.341661 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d665ab22-86aa-4973-bce7-1ce3c610fd7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:46.351642 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91916e5a-b0cd-4ee4-b948-5df438128954 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:46.380632 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-d665ab22-86aa-4973-bce7-1ce3c610fd7e None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:33:46.382107 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-d665ab22-86aa-4973-bce7-1ce3c610fd7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:33:46.387478 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf837b6b-7085-4ca2-97ea-36b1a1c70f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:46.387478 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf837b6b-7085-4ca2-97ea-36b1a1c70f10 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:46.387969 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf837b6b-7085-4ca2-97ea-36b1a1c70f10 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:46.388364 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf837b6b-7085-4ca2-97ea-36b1a1c70f10 None admin] PATH_INFO: `/v3/roles/service` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:46.399407 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf837b6b-7085-4ca2-97ea-36b1a1c70f10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:33:46.404303 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-bf837b6b-7085-4ca2-97ea-36b1a1c70f10 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 10:33:46.404303 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:46.404303 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:46.404303 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:46.404303 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:46.404303 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:46.404303 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:46.404303 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:46.404303 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:46.404303 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:46.404303 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:46.404303 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:46.404303 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:33:46.404303 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:33:46.404303 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:33:46.404303 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:33:46.405324 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:33:46.405324 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:33:46.405324 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:33:46.405324 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:33:46.405324 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:46.405324 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:46.405324 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:46.405324 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:46.405324 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:46.405324 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:46.405324 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:46.405324 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:46.405324 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:46.405324 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:46.405324 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:46.405324 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:46.406280 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:46.406280 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:46.406280 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:46.406280 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:46.406280 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:33:46.406280 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:33:46.406280 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:33:46.406280 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:33:46.406280 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:33:46.406280 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:33:46.406280 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 10:33:46.406280 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:33:46.408477 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 244/488] 10.222.0.15 () {56 vars in 1102 bytes} [Sun Sep 11 10:33:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:46.414798 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf837b6b-7085-4ca2-97ea-36b1a1c70f10 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:46.465690 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:46.499515 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 245/489] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:33:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 10:33:46.499515 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d665ab22-86aa-4973-bce7-1ce3c610fd7e None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:46.582058 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf837b6b-7085-4ca2-97ea-36b1a1c70f10 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:46.602935 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:46.623006 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d665ab22-86aa-4973-bce7-1ce3c610fd7e None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:46.624592 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630f098d-b7b6-4070-a195-ac1d833775dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:46.625457 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630f098d-b7b6-4070-a195-ac1d833775dc None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:46.625928 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630f098d-b7b6-4070-a195-ac1d833775dc None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:46.626627 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630f098d-b7b6-4070-a195-ac1d833775dc None admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:46.627853 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630f098d-b7b6-4070-a195-ac1d833775dc None admin] RBAC: Adding query filter params (name=service) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:46.628363 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630f098d-b7b6-4070-a195-ac1d833775dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:46.629983 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630f098d-b7b6-4070-a195-ac1d833775dc None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:46.640599 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 245/490] 10.222.0.15 () {56 vars in 1103 bytes} [Sun Sep 11 10:33:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:46.641752 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630f098d-b7b6-4070-a195-ac1d833775dc None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:46.671943 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f262ca-c90f-4b8c-be41-ead6058b2c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:46.673263 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f262ca-c90f-4b8c-be41-ead6058b2c65 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:46.673779 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f262ca-c90f-4b8c-be41-ead6058b2c65 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:46.674502 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f262ca-c90f-4b8c-be41-ead6058b2c65 None admin] PATH_INFO: `/v3/roles/service` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:46.692325 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f262ca-c90f-4b8c-be41-ead6058b2c65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:33:46.700503 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-32f262ca-c90f-4b8c-be41-ead6058b2c65 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 10:33:46.700503 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:46.700503 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:46.700503 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:46.700503 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:46.700503 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:46.700503 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:46.700503 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:46.700503 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:46.700503 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:46.700503 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:46.700503 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:46.700503 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:33:46.700503 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:33:46.700503 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:33:46.700503 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:33:46.701380 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:33:46.701380 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:33:46.701380 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:33:46.701380 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:33:46.701380 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:46.701380 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:46.701380 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:46.701380 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:46.701380 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:46.701380 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:46.701380 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:46.701380 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:46.701380 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:46.701380 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:46.701380 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:46.701380 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:46.702216 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:46.702216 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:46.702216 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:46.702216 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:46.702216 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:33:46.702216 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:33:46.702216 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:33:46.702216 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:33:46.702216 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:33:46.702216 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:33:46.702216 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 10:33:46.702216 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:33:46.705176 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 246/491] 10.222.0.15 () {56 vars in 1102 bytes} [Sun Sep 11 10:33:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:46.706422 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f262ca-c90f-4b8c-be41-ead6058b2c65 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:46.768133 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:46.771949 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630f098d-b7b6-4070-a195-ac1d833775dc None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:46.813989 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ba2459-6ebe-4f8c-a821-f1089684ee1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:46.814755 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ba2459-6ebe-4f8c-a821-f1089684ee1b None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:46.815028 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ba2459-6ebe-4f8c-a821-f1089684ee1b None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:46.815303 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ba2459-6ebe-4f8c-a821-f1089684ee1b None admin] PATH_INFO: `/v3/roles/service` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:46.828439 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ba2459-6ebe-4f8c-a821-f1089684ee1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:33:46.829972 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f262ca-c90f-4b8c-be41-ead6058b2c65 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:46.834049 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-33ba2459-6ebe-4f8c-a821-f1089684ee1b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 10:33:46.834049 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:46.834049 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:46.834049 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:46.834049 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:46.834049 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:46.834049 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:46.834049 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:46.834049 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:46.834049 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:46.834049 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:46.834049 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:46.834049 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:33:46.834049 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:33:46.834049 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:33:46.834049 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:33:46.834897 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:33:46.834897 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:33:46.834897 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:33:46.834897 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:33:46.834897 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:46.834897 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:46.834897 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:46.834897 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:46.834897 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:46.834897 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:46.834897 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:46.834897 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:46.834897 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:46.834897 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:46.834897 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:46.834897 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:46.836091 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:46.836091 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:46.836091 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:46.836091 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:46.836091 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:33:46.836091 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:33:46.836091 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:33:46.836091 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:33:46.836091 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:33:46.836091 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:33:46.836091 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 10:33:46.836091 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:33:46.836091 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 246/492] 10.222.0.15 () {56 vars in 1102 bytes} [Sun Sep 11 10:33:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:46.836746 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ba2459-6ebe-4f8c-a821-f1089684ee1b None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:46.870890 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d08960-e88d-43c6-b794-e0096131d944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:46.872121 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d08960-e88d-43c6-b794-e0096131d944 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:46.872543 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d08960-e88d-43c6-b794-e0096131d944 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:46.872953 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d08960-e88d-43c6-b794-e0096131d944 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:46.883412 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:46.886151 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d08960-e88d-43c6-b794-e0096131d944 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:46.887658 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d08960-e88d-43c6-b794-e0096131d944 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:46.902638 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-c3d08960-e88d-43c6-b794-e0096131d944 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:33:46.902638 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:46.902638 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:46.902638 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:46.902638 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:46.902638 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:46.902638 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:46.902638 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:46.902638 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:46.902638 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:46.902638 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:46.902638 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:46.902638 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:33:46.902638 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:33:46.902638 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:33:46.902638 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:33:46.903724 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:46.903724 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:46.903724 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:46.903724 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:46.903724 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:46.903724 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:46.903724 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:46.903724 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:46.903724 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:46.903724 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:46.903724 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:46.903724 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:46.903724 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:46.903724 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:46.903724 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:46.903724 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:46.903724 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:33:46.904623 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:33:46.904623 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:33:46.904623 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:33:46.904623 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 247/493] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:33:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:46.904623 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d08960-e88d-43c6-b794-e0096131d944 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:46.954908 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ba2459-6ebe-4f8c-a821-f1089684ee1b None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:46.989268 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30d0f6f-af63-4c79-be4c-897a05f5cf05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:46.989880 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d0f6f-af63-4c79-be4c-897a05f5cf05 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:46.990154 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d0f6f-af63-4c79-be4c-897a05f5cf05 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:46.990419 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d0f6f-af63-4c79-be4c-897a05f5cf05 None admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:46.991149 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30d0f6f-af63-4c79-be4c-897a05f5cf05 None admin] RBAC: Adding query filter params (name=service) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:46.991472 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30d0f6f-af63-4c79-be4c-897a05f5cf05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:46.993421 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30d0f6f-af63-4c79-be4c-897a05f5cf05 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:47.002656 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 247/494] 10.222.0.15 () {56 vars in 1103 bytes} [Sun Sep 11 10:33:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:47.003466 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30d0f6f-af63-4c79-be4c-897a05f5cf05 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:47.012572 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:47.024385 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d08960-e88d-43c6-b794-e0096131d944 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:47.067169 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13332dba-45f7-4df9-8ef2-480a9150d326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:47.067833 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13332dba-45f7-4df9-8ef2-480a9150d326 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:47.068112 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13332dba-45f7-4df9-8ef2-480a9150d326 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:47.068582 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13332dba-45f7-4df9-8ef2-480a9150d326 None admin] PATH_INFO: `/v3/roles/service` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:47.078856 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13332dba-45f7-4df9-8ef2-480a9150d326 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:33:47.084110 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-13332dba-45f7-4df9-8ef2-480a9150d326 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 10:33:47.084110 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:47.084110 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:47.084110 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:47.084110 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:47.084110 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:47.084110 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:47.084110 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:47.084110 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:47.084110 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:47.084110 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:47.084110 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:47.084110 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:33:47.084110 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:33:47.084110 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:33:47.084110 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:33:47.085102 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:33:47.085102 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:33:47.085102 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:33:47.085102 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:33:47.085102 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:47.085102 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:47.085102 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:47.085102 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:47.085102 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:47.085102 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:47.085102 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:47.085102 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:47.085102 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:47.085102 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:47.085102 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:47.085102 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:47.086588 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:47.086588 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:47.086588 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:47.086588 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:47.086588 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:33:47.086588 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:33:47.086588 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:33:47.086588 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:33:47.086588 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:33:47.086588 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:33:47.086588 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 10:33:47.086588 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:33:47.086588 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 248/495] 10.222.0.15 () {56 vars in 1102 bytes} [Sun Sep 11 10:33:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:47.087261 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13332dba-45f7-4df9-8ef2-480a9150d326 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:47.122683 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30d0f6f-af63-4c79-be4c-897a05f5cf05 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:47.124379 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:47.158641 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d828978-4bd4-4169-92f3-a407b8a95452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:47.159165 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d828978-4bd4-4169-92f3-a407b8a95452 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:47.159468 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d828978-4bd4-4169-92f3-a407b8a95452 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:47.159748 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d828978-4bd4-4169-92f3-a407b8a95452 None admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:47.160407 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d828978-4bd4-4169-92f3-a407b8a95452 None admin] RBAC: Adding query filter params (name=service) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:47.160715 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d828978-4bd4-4169-92f3-a407b8a95452 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:47.162097 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d828978-4bd4-4169-92f3-a407b8a95452 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:47.170383 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 248/496] 10.222.0.15 () {56 vars in 1103 bytes} [Sun Sep 11 10:33:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:47.171357 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d828978-4bd4-4169-92f3-a407b8a95452 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:47.196153 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13332dba-45f7-4df9-8ef2-480a9150d326 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:47.229278 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2a1de9-dc1f-44fd-9385-2ee66657f586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:47.229884 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a1de9-dc1f-44fd-9385-2ee66657f586 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:47.230167 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a1de9-dc1f-44fd-9385-2ee66657f586 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:47.230440 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a1de9-dc1f-44fd-9385-2ee66657f586 None admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:47.231078 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2a1de9-dc1f-44fd-9385-2ee66657f586 None admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:47.231378 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2a1de9-dc1f-44fd-9385-2ee66657f586 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:47.232791 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2a1de9-dc1f-44fd-9385-2ee66657f586 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:47.244367 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 249/497] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:33:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:47.245210 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2a1de9-dc1f-44fd-9385-2ee66657f586 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:47.259140 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:47.276595 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d828978-4bd4-4169-92f3-a407b8a95452 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:47.311173 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3fd01a-ceb9-4716-8d37-c903356c9fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:47.311830 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3fd01a-ceb9-4716-8d37-c903356c9fe5 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:47.312110 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3fd01a-ceb9-4716-8d37-c903356c9fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:47.312379 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3fd01a-ceb9-4716-8d37-c903356c9fe5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:47.323492 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3fd01a-ceb9-4716-8d37-c903356c9fe5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:47.324939 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3fd01a-ceb9-4716-8d37-c903356c9fe5 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:47.339319 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-ee3fd01a-ceb9-4716-8d37-c903356c9fe5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:33:47.339319 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:47.339319 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:47.339319 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:47.339319 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:47.339319 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:47.339319 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:47.339319 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:47.339319 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:47.339319 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:47.339319 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:47.339319 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:47.339319 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:33:47.339319 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:33:47.339319 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:33:47.339319 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:33:47.340310 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:47.340310 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:47.340310 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:47.340310 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:47.340310 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:47.340310 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:47.340310 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:47.340310 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:47.340310 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:47.340310 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:47.340310 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:47.340310 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:47.340310 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:47.340310 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:47.340310 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:47.340310 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:47.340310 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:33:47.341152 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:33:47.341152 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:33:47.341152 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:33:47.341152 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 249/498] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:33:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:47.341558 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f37d86-cf50-4299-bab5-16b803a1f3c1 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:47.341846 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f37d86-cf50-4299-bab5-16b803a1f3c1 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:47.342120 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f37d86-cf50-4299-bab5-16b803a1f3c1 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:47.343320 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 250/499] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:33:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:33:47.344031 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f37d86-cf50-4299-bab5-16b803a1f3c1 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:47.349563 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2a1de9-dc1f-44fd-9385-2ee66657f586 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:47.363588 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:47.385594 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cda4c0-fc90-4861-89e4-ed0b35797f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:47.386197 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cda4c0-fc90-4861-89e4-ed0b35797f15 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:47.386478 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cda4c0-fc90-4861-89e4-ed0b35797f15 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:47.386750 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cda4c0-fc90-4861-89e4-ed0b35797f15 None admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:47.387403 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cda4c0-fc90-4861-89e4-ed0b35797f15 None admin] RBAC: Adding query filter params (name=service) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:47.387739 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cda4c0-fc90-4861-89e4-ed0b35797f15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:47.389115 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cda4c0-fc90-4861-89e4-ed0b35797f15 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:47.397058 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 250/500] 10.222.0.15 () {56 vars in 1103 bytes} [Sun Sep 11 10:33:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:47.397842 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cda4c0-fc90-4861-89e4-ed0b35797f15 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:47.452777 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f37d86-cf50-4299-bab5-16b803a1f3c1 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:47.473452 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:47.487437 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4e3c9c-4e63-461e-93f4-4b3451fd3991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:47.488085 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e3c9c-4e63-461e-93f4-4b3451fd3991 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:47.488395 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e3c9c-4e63-461e-93f4-4b3451fd3991 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:47.488671 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e3c9c-4e63-461e-93f4-4b3451fd3991 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:47.499976 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cda4c0-fc90-4861-89e4-ed0b35797f15 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:47.500105 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4e3c9c-4e63-461e-93f4-4b3451fd3991 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:47.501419 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4e3c9c-4e63-461e-93f4-4b3451fd3991 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:47.515130 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-9d4e3c9c-4e63-461e-93f4-4b3451fd3991 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:33:47.515130 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:47.515130 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:47.515130 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:47.515130 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:47.515130 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:47.515130 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:47.515130 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:47.515130 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:47.515130 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:47.515130 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:47.515130 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:47.515130 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:33:47.515130 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:33:47.515130 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:33:47.515130 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:33:47.516062 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:47.516062 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:47.516062 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:47.516062 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:47.516062 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:47.516062 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:47.516062 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:47.516062 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:47.516062 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:47.516062 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:47.516062 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:47.516062 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:47.516062 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:47.516062 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:47.516062 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:47.516062 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:47.516062 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:33:47.516919 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:33:47.516919 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:33:47.516919 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:33:47.516919 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 251/501] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:33:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:47.517151 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4e3c9c-4e63-461e-93f4-4b3451fd3991 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:47.532735 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0ce92e-15fa-4e46-840c-2f562b351dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:47.533370 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ce92e-15fa-4e46-840c-2f562b351dbe None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:47.533653 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ce92e-15fa-4e46-840c-2f562b351dbe None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:47.533930 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ce92e-15fa-4e46-840c-2f562b351dbe None admin] PATH_INFO: `/v3/users/glance` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:47.542023 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0ce92e-15fa-4e46-840c-2f562b351dbe None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:47.543497 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0ce92e-15fa-4e46-840c-2f562b351dbe None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:47.555892 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-aa0ce92e-15fa-4e46-840c-2f562b351dbe None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 11 10:33:47.555892 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:47.555892 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:47.555892 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:47.555892 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:47.555892 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:47.555892 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:47.555892 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:47.555892 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:47.555892 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:47.555892 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:47.555892 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:47.555892 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 10:33:47.555892 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 10:33:47.555892 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 10:33:47.555892 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 10:33:47.556889 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:47.556889 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:47.556889 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 10:33:47.556889 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:33:47.556889 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 10:33:47.556889 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:33:47.556889 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:47.556889 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:47.556889 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:47.556889 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:47.556889 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:47.556889 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:47.556889 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:47.556889 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:47.556889 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:47.556889 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:47.556889 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:47.556889 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:47.557793 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:47.557793 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:47.557793 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 10:33:47.557793 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 10:33:47.557793 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 10:33:47.557793 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 10:33:47.557793 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 10:33:47.557793 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 10:33:47.557793 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 11 10:33:47.557793 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:33:47.557793 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 251/502] 10.222.0.15 () {56 vars in 1099 bytes} [Sun Sep 11 10:33:47 2022] GET /identity/v3/users/glance => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:47.558378 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeabb48-0b04-45ad-8f0c-7cbfbc2f7010 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:47.558443 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeabb48-0b04-45ad-8f0c-7cbfbc2f7010 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:47.558918 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeabb48-0b04-45ad-8f0c-7cbfbc2f7010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:47.576510 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-daeabb48-0b04-45ad-8f0c-7cbfbc2f7010 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:33:47.577707 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-daeabb48-0b04-45ad-8f0c-7cbfbc2f7010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:33:47.591118 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:47.624933 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4e3c9c-4e63-461e-93f4-4b3451fd3991 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:47.644968 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 252/503] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:33:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:33:47.645799 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeabb48-0b04-45ad-8f0c-7cbfbc2f7010 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:47.658225 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c53261-d1ed-4bc5-8e36-7671b5eb02d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:47.658836 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c53261-d1ed-4bc5-8e36-7671b5eb02d4 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:47.659120 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c53261-d1ed-4bc5-8e36-7671b5eb02d4 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:47.659394 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c53261-d1ed-4bc5-8e36-7671b5eb02d4 None admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:47.660148 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c53261-d1ed-4bc5-8e36-7671b5eb02d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:47.660575 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c53261-d1ed-4bc5-8e36-7671b5eb02d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:47.661958 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c53261-d1ed-4bc5-8e36-7671b5eb02d4 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:47.676869 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 252/504] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:33:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:47.676869 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c53261-d1ed-4bc5-8e36-7671b5eb02d4 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:47.719843 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:47.760001 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeabb48-0b04-45ad-8f0c-7cbfbc2f7010 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:47.784491 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c53261-d1ed-4bc5-8e36-7671b5eb02d4 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:47.795271 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cb7eb8-0be6-4509-bfaf-08d3070b0d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:47.796276 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cb7eb8-0be6-4509-bfaf-08d3070b0d0d None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:47.796560 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cb7eb8-0be6-4509-bfaf-08d3070b0d0d None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:47.796986 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cb7eb8-0be6-4509-bfaf-08d3070b0d0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:47.807751 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cb7eb8-0be6-4509-bfaf-08d3070b0d0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:47.809197 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cb7eb8-0be6-4509-bfaf-08d3070b0d0d None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:47.812846 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:47.822790 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297b0574-9a92-49f1-8afc-10d46d6e2a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:47.823397 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b0574-9a92-49f1-8afc-10d46d6e2a9c None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:47.823708 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b0574-9a92-49f1-8afc-10d46d6e2a9c None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:47.823981 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b0574-9a92-49f1-8afc-10d46d6e2a9c None admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:47.824621 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297b0574-9a92-49f1-8afc-10d46d6e2a9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:47.824924 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297b0574-9a92-49f1-8afc-10d46d6e2a9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:47.826017 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-a0cb7eb8-0be6-4509-bfaf-08d3070b0d0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:33:47.826017 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:47.826017 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:47.826017 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:47.826017 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:47.826017 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:47.826017 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:47.826017 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:47.826017 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:47.826017 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:47.826017 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:47.826017 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:47.826017 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:33:47.826017 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:33:47.826017 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:33:47.826017 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:33:47.826845 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:47.826845 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:47.826845 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:47.826845 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:47.826845 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:47.826845 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:47.826845 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:47.826845 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:47.826845 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:47.826845 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:47.826845 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:47.826845 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:47.826845 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:47.826845 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:47.826845 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:47.826845 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:47.826845 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:33:47.828467 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:33:47.828467 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:33:47.828467 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:33:47.828654 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297b0574-9a92-49f1-8afc-10d46d6e2a9c None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:47.828734 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 253/505] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:33:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:47.828734 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cb7eb8-0be6-4509-bfaf-08d3070b0d0d None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:47.843536 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 253/506] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:33:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:47.844164 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297b0574-9a92-49f1-8afc-10d46d6e2a9c None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:47.933582 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cb7eb8-0be6-4509-bfaf-08d3070b0d0d None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:47.950023 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:47.950726 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297b0574-9a92-49f1-8afc-10d46d6e2a9c None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:47.968578 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5438a8f-4751-4944-9300-90798c3f47ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:47.969176 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5438a8f-4751-4944-9300-90798c3f47ae None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:47.969454 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5438a8f-4751-4944-9300-90798c3f47ae None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:47.969723 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5438a8f-4751-4944-9300-90798c3f47ae None admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:47.970432 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5438a8f-4751-4944-9300-90798c3f47ae None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:47.970729 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5438a8f-4751-4944-9300-90798c3f47ae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:47.972181 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5438a8f-4751-4944-9300-90798c3f47ae None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:47.985110 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80b67cb-f358-4ba9-9930-d2a70462d54d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:47.985734 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80b67cb-f358-4ba9-9930-d2a70462d54d None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:47.986014 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80b67cb-f358-4ba9-9930-d2a70462d54d None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:47.986284 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80b67cb-f358-4ba9-9930-d2a70462d54d None admin] PATH_INFO: `/v3/users/nova` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:47.993732 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80b67cb-f358-4ba9-9930-d2a70462d54d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:47.995149 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80b67cb-f358-4ba9-9930-d2a70462d54d None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:48.008100 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-d80b67cb-f358-4ba9-9930-d2a70462d54d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 11 10:33:48.008100 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:48.008100 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:48.008100 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:48.008100 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:48.008100 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:48.008100 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:48.008100 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:48.008100 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:48.008100 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:48.008100 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:48.008100 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:48.008100 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 10:33:48.008100 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 10:33:48.008100 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 10:33:48.008100 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 10:33:48.009117 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:48.009117 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:48.009117 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 10:33:48.009117 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:33:48.009117 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 10:33:48.009117 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:33:48.009117 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:48.009117 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:48.009117 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:48.009117 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:48.009117 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:48.009117 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:48.009117 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:48.009117 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:48.009117 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:48.009117 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:48.009117 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:48.009117 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:48.010017 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:48.010017 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:48.010017 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 10:33:48.010017 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 10:33:48.010017 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 10:33:48.010017 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 10:33:48.010017 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 10:33:48.010017 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 10:33:48.010017 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 11 10:33:48.010017 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:33:48.010017 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 254/507] 10.222.0.15 () {56 vars in 1093 bytes} [Sun Sep 11 10:33:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:48.010017 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80b67cb-f358-4ba9-9930-d2a70462d54d None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:48.014062 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 254/508] 10.222.0.15 () {56 vars in 1137 bytes} [Sun Sep 11 10:33:47 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:48.014062 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5438a8f-4751-4944-9300-90798c3f47ae None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:48.061025 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:48.110257 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5438a8f-4751-4944-9300-90798c3f47ae None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:48.112444 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80b67cb-f358-4ba9-9930-d2a70462d54d None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:48.143577 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a9759b-6474-4bd7-9e72-31f85ea317cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:48.144199 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a9759b-6474-4bd7-9e72-31f85ea317cb None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:48.144478 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a9759b-6474-4bd7-9e72-31f85ea317cb None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:48.144749 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a9759b-6474-4bd7-9e72-31f85ea317cb None admin] PATH_INFO: `/v3/users/cinder` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:48.145924 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bee11c8-7960-4e27-a1f8-4936614ca5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:48.146528 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bee11c8-7960-4e27-a1f8-4936614ca5c3 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:48.146810 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bee11c8-7960-4e27-a1f8-4936614ca5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:48.147085 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bee11c8-7960-4e27-a1f8-4936614ca5c3 None admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:48.147752 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bee11c8-7960-4e27-a1f8-4936614ca5c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:48.148084 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bee11c8-7960-4e27-a1f8-4936614ca5c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:48.149471 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bee11c8-7960-4e27-a1f8-4936614ca5c3 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:48.151851 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a9759b-6474-4bd7-9e72-31f85ea317cb None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:48.153283 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a9759b-6474-4bd7-9e72-31f85ea317cb None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:48.160873 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 255/509] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:33:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:48.161657 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bee11c8-7960-4e27-a1f8-4936614ca5c3 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:48.165189 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-24a9759b-6474-4bd7-9e72-31f85ea317cb None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 11 10:33:48.165189 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:48.165189 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:48.165189 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:48.165189 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:48.165189 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:48.165189 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:48.165189 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:48.165189 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:48.165189 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:48.165189 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:48.165189 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:48.165189 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 10:33:48.165189 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 10:33:48.165189 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 10:33:48.165189 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 10:33:48.166074 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:48.166074 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:48.166074 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 10:33:48.166074 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:33:48.166074 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 10:33:48.166074 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:33:48.166074 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:48.166074 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:48.166074 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:48.166074 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:48.166074 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:48.166074 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:48.166074 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:48.166074 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:48.166074 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:48.166074 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:48.166074 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:48.166074 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:48.166968 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:48.166968 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:48.166968 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 10:33:48.166968 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 10:33:48.166968 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 10:33:48.166968 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 10:33:48.166968 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 10:33:48.166968 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 10:33:48.166968 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 11 10:33:48.166968 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:33:48.166968 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 255/510] 10.222.0.15 () {56 vars in 1099 bytes} [Sun Sep 11 10:33:48 2022] GET /identity/v3/users/cinder => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:48.167567 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a9759b-6474-4bd7-9e72-31f85ea317cb None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:48.202036 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:48.264947 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bee11c8-7960-4e27-a1f8-4936614ca5c3 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:48.267194 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a9759b-6474-4bd7-9e72-31f85ea317cb None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:48.274504 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:48.303212 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b718152-ebf6-494c-82da-2dc96b0c48b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:48.303870 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b718152-ebf6-494c-82da-2dc96b0c48b1 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:48.304583 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228a629d-030f-4ab5-af6e-8d67132091ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:48.304739 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b718152-ebf6-494c-82da-2dc96b0c48b1 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:48.304937 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b718152-ebf6-494c-82da-2dc96b0c48b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:48.305189 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228a629d-030f-4ab5-af6e-8d67132091ac None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:48.305468 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228a629d-030f-4ab5-af6e-8d67132091ac None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:48.306198 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228a629d-030f-4ab5-af6e-8d67132091ac None admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:48.306903 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228a629d-030f-4ab5-af6e-8d67132091ac None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:48.307203 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228a629d-030f-4ab5-af6e-8d67132091ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:48.308690 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228a629d-030f-4ab5-af6e-8d67132091ac None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:48.316333 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b718152-ebf6-494c-82da-2dc96b0c48b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:48.317785 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b718152-ebf6-494c-82da-2dc96b0c48b1 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:48.326509 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 256/511] 10.222.0.15 () {56 vars in 1133 bytes} [Sun Sep 11 10:33:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:48.327378 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228a629d-030f-4ab5-af6e-8d67132091ac None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:48.337925 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-3b718152-ebf6-494c-82da-2dc96b0c48b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:33:48.337925 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:48.337925 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:48.337925 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:48.337925 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:48.337925 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:48.337925 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:48.337925 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:48.337925 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:48.337925 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:48.337925 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:48.337925 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:48.337925 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:33:48.337925 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:33:48.337925 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:33:48.337925 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:33:48.338842 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:48.338842 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:48.338842 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:48.338842 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:48.338842 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:48.338842 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:48.338842 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:48.338842 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:48.338842 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:48.338842 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:48.338842 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:48.338842 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:48.338842 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:48.338842 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:48.338842 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:48.338842 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:48.338842 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:33:48.339728 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:33:48.339728 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:33:48.339728 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:33:48.339728 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 256/512] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:33:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:48.339944 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b718152-ebf6-494c-82da-2dc96b0c48b1 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:48.420681 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:48.429983 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228a629d-030f-4ab5-af6e-8d67132091ac None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:48.441189 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b718152-ebf6-494c-82da-2dc96b0c48b1 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:48.463629 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a234573f-8181-4af8-a644-f6e23a3d9ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:48.464234 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a234573f-8181-4af8-a644-f6e23a3d9ad5 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:48.464511 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a234573f-8181-4af8-a644-f6e23a3d9ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:48.464785 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a234573f-8181-4af8-a644-f6e23a3d9ad5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:48.473967 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58bc027-307b-480b-a725-a5bd7d8746cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:48.474573 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58bc027-307b-480b-a725-a5bd7d8746cb None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:48.474856 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58bc027-307b-480b-a725-a5bd7d8746cb None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:48.475119 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58bc027-307b-480b-a725-a5bd7d8746cb None admin] PATH_INFO: `/v3/users/neutron` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:48.482632 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58bc027-307b-480b-a725-a5bd7d8746cb None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:48.484376 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58bc027-307b-480b-a725-a5bd7d8746cb None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:48.495801 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-c58bc027-307b-480b-a725-a5bd7d8746cb None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 11 10:33:48.495801 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:48.495801 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:48.495801 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:48.495801 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:48.495801 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:48.495801 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:48.495801 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:48.495801 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:48.495801 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:48.495801 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:48.495801 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:48.495801 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 10:33:48.495801 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 10:33:48.495801 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 10:33:48.495801 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 10:33:48.496887 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:48.496887 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:48.496887 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 10:33:48.496887 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:33:48.496887 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 10:33:48.496887 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:33:48.496887 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:48.496887 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:48.496887 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:48.496887 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:48.496887 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:48.496887 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:48.496887 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:48.496887 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:48.496887 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:48.496887 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:48.496887 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:48.496887 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:48.497819 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:48.497819 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:48.497819 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 10:33:48.497819 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 10:33:48.497819 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 10:33:48.497819 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 10:33:48.497819 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 10:33:48.497819 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 10:33:48.497819 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 11 10:33:48.497819 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:33:48.501569 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 257/513] 10.222.0.15 () {56 vars in 1102 bytes} [Sun Sep 11 10:33:48 2022] GET /identity/v3/users/neutron => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:48.502302 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58bc027-307b-480b-a725-a5bd7d8746cb None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:48.523697 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:48.533093 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a234573f-8181-4af8-a644-f6e23a3d9ad5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:48.534553 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a234573f-8181-4af8-a644-f6e23a3d9ad5 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:48.603106 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58bc027-307b-480b-a725-a5bd7d8746cb None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:48.621941 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:48.635805 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 257/514] 10.222.0.15 () {58 vars in 1282 bytes} [Sun Sep 11 10:33:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 10:33:48.635805 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a234573f-8181-4af8-a644-f6e23a3d9ad5 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:48.642492 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82d7fd1-a78f-4806-aeae-991c0a997f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:48.643405 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82d7fd1-a78f-4806-aeae-991c0a997f1d None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:48.643718 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82d7fd1-a78f-4806-aeae-991c0a997f1d None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:48.644138 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82d7fd1-a78f-4806-aeae-991c0a997f1d None admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:48.644854 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82d7fd1-a78f-4806-aeae-991c0a997f1d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:48.645151 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82d7fd1-a78f-4806-aeae-991c0a997f1d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:48.646555 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82d7fd1-a78f-4806-aeae-991c0a997f1d None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:48.687371 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 258/515] 10.222.0.15 () {56 vars in 1137 bytes} [Sun Sep 11 10:33:48 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:48.688314 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82d7fd1-a78f-4806-aeae-991c0a997f1d None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:48.730374 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a234573f-8181-4af8-a644-f6e23a3d9ad5 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:48.765657 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89000380-9d6e-481f-9e4d-51c89fb1a677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:48.766276 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89000380-9d6e-481f-9e4d-51c89fb1a677 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:48.766555 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89000380-9d6e-481f-9e4d-51c89fb1a677 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:48.767238 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89000380-9d6e-481f-9e4d-51c89fb1a677 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:48.775766 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:48.794422 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82d7fd1-a78f-4806-aeae-991c0a997f1d None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:48.831473 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1f870b-1abc-466a-b3e1-45e6eac9ccf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:48.832492 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1f870b-1abc-466a-b3e1-45e6eac9ccf3 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:48.833184 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1f870b-1abc-466a-b3e1-45e6eac9ccf3 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:48.833614 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1f870b-1abc-466a-b3e1-45e6eac9ccf3 None admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:48.834647 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:48.835121 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1f870b-1abc-466a-b3e1-45e6eac9ccf3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:48.835121 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1f870b-1abc-466a-b3e1-45e6eac9ccf3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:48.836412 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1f870b-1abc-466a-b3e1-45e6eac9ccf3 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:48.853970 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 259/516] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:33:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:48.853970 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1f870b-1abc-466a-b3e1-45e6eac9ccf3 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:48.856731 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89000380-9d6e-481f-9e4d-51c89fb1a677 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:48.858292 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89000380-9d6e-481f-9e4d-51c89fb1a677 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:48.963882 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 258/517] 10.222.0.15 () {58 vars in 1282 bytes} [Sun Sep 11 10:33:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 10:33:48.964655 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89000380-9d6e-481f-9e4d-51c89fb1a677 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:48.968167 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1f870b-1abc-466a-b3e1-45e6eac9ccf3 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:49.009061 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ee9786-e076-4bb7-beaa-a43939bff8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:49.009674 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee9786-e076-4bb7-beaa-a43939bff8ee None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:49.009961 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee9786-e076-4bb7-beaa-a43939bff8ee None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:49.010356 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee9786-e076-4bb7-beaa-a43939bff8ee None admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:49.011084 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ee9786-e076-4bb7-beaa-a43939bff8ee None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:49.011389 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ee9786-e076-4bb7-beaa-a43939bff8ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:49.012985 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ee9786-e076-4bb7-beaa-a43939bff8ee None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:49.024472 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:49.035584 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 260/518] 10.222.0.15 () {56 vars in 1139 bytes} [Sun Sep 11 10:33:48 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:49.036703 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ee9786-e076-4bb7-beaa-a43939bff8ee None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:49.066197 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:49.075845 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89000380-9d6e-481f-9e4d-51c89fb1a677 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:49.110377 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2436329-9f8a-463d-a60a-9c3df8ceeb23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:49.111015 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2436329-9f8a-463d-a60a-9c3df8ceeb23 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:49.111295 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2436329-9f8a-463d-a60a-9c3df8ceeb23 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:49.111600 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2436329-9f8a-463d-a60a-9c3df8ceeb23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:49.122115 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2436329-9f8a-463d-a60a-9c3df8ceeb23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:49.123701 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2436329-9f8a-463d-a60a-9c3df8ceeb23 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:49.138634 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-e2436329-9f8a-463d-a60a-9c3df8ceeb23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:33:49.138634 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:49.138634 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:49.138634 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:49.138634 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:49.138634 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:49.138634 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:49.138634 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:49.138634 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:49.138634 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:49.138634 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:49.138634 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:49.138634 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:33:49.138634 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:33:49.138634 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:33:49.138634 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:33:49.139746 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:49.139746 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:49.139746 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:49.139746 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:49.139746 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:49.139746 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:49.139746 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:49.139746 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:49.139746 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:49.139746 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:49.139746 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:49.139746 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:49.139746 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:49.139746 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:49.139746 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:49.139746 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:49.139746 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:33:49.140627 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:33:49.140627 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:33:49.140627 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:33:49.140627 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 259/519] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:33:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:49.140627 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2436329-9f8a-463d-a60a-9c3df8ceeb23 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:49.142974 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ee9786-e076-4bb7-beaa-a43939bff8ee None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:49.177280 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5dc741-42cc-4142-944b-38e1249499b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:49.177877 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5dc741-42cc-4142-944b-38e1249499b2 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:49.178159 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5dc741-42cc-4142-944b-38e1249499b2 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:49.178430 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5dc741-42cc-4142-944b-38e1249499b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:49.239457 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:49.239610 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2436329-9f8a-463d-a60a-9c3df8ceeb23 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:49.247007 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5dc741-42cc-4142-944b-38e1249499b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:49.248522 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5dc741-42cc-4142-944b-38e1249499b2 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:49.273754 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502103a6-c6d7-4325-92ea-3fb9133e6f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:49.274362 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502103a6-c6d7-4325-92ea-3fb9133e6f00 None admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:49.274641 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502103a6-c6d7-4325-92ea-3fb9133e6f00 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:49.274924 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502103a6-c6d7-4325-92ea-3fb9133e6f00 None admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:49.275593 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502103a6-c6d7-4325-92ea-3fb9133e6f00 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:49.277025 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502103a6-c6d7-4325-92ea-3fb9133e6f00 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:49.294744 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:49.299865 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 260/520] 10.222.0.15 () {60 vars in 1132 bytes} [Sun Sep 11 10:33:49 2022] POST /identity/v3/users => generated 247 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:33:49.300682 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502103a6-c6d7-4325-92ea-3fb9133e6f00 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:49.361809 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 261/521] 10.222.0.15 () {58 vars in 1282 bytes} [Sun Sep 11 10:33:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 10:33:49.362977 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5dc741-42cc-4142-944b-38e1249499b2 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:49.440465 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502103a6-c6d7-4325-92ea-3fb9133e6f00 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:49.478758 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:49.480638 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091e2523-3de9-4d09-96d6-1736e888a45b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:49.481439 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e2523-3de9-4d09-96d6-1736e888a45b None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:49.482008 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e2523-3de9-4d09-96d6-1736e888a45b None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:49.482008 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e2523-3de9-4d09-96d6-1736e888a45b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:49.486800 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5dc741-42cc-4142-944b-38e1249499b2 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:49.495358 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091e2523-3de9-4d09-96d6-1736e888a45b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:49.496475 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091e2523-3de9-4d09-96d6-1736e888a45b None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:49.512094 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-091e2523-3de9-4d09-96d6-1736e888a45b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:33:49.512094 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:49.512094 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:49.512094 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:49.512094 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:49.512094 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:49.512094 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:49.512094 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:49.512094 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:49.512094 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:49.512094 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:49.512094 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:49.512094 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:33:49.512094 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:33:49.512094 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:33:49.512094 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:33:49.513038 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:49.513038 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:49.513038 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:49.513038 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:49.513038 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:49.513038 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:49.513038 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:49.513038 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:49.513038 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:49.513038 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:49.513038 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:49.513038 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:49.513038 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:49.513038 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:49.513038 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:49.513038 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:49.513038 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:33:49.513865 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:33:49.513865 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:33:49.513865 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:33:49.513865 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 261/522] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:33:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:49.514504 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091e2523-3de9-4d09-96d6-1736e888a45b None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:49.527906 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee1c103-ccd8-4a7b-9a4f-d94eef0dde24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:49.528521 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee1c103-ccd8-4a7b-9a4f-d94eef0dde24 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:49.528800 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee1c103-ccd8-4a7b-9a4f-d94eef0dde24 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:49.529081 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee1c103-ccd8-4a7b-9a4f-d94eef0dde24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:49.572111 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:49.606927 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee1c103-ccd8-4a7b-9a4f-d94eef0dde24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:49.608376 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee1c103-ccd8-4a7b-9a4f-d94eef0dde24 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:49.624016 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091e2523-3de9-4d09-96d6-1736e888a45b None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:49.665439 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec39d092-b9e5-467c-97d3-549e11fa522d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:49.666333 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec39d092-b9e5-467c-97d3-549e11fa522d None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:49.666617 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec39d092-b9e5-467c-97d3-549e11fa522d None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:49.667042 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec39d092-b9e5-467c-97d3-549e11fa522d None admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:49.668042 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec39d092-b9e5-467c-97d3-549e11fa522d None admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:49.668358 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec39d092-b9e5-467c-97d3-549e11fa522d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:49.670261 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec39d092-b9e5-467c-97d3-549e11fa522d None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:49.689900 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 262/523] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:33:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:49.689900 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec39d092-b9e5-467c-97d3-549e11fa522d None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:49.708540 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:49.720831 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 262/524] 10.222.0.15 () {58 vars in 1282 bytes} [Sun Sep 11 10:33:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 10:33:49.720831 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee1c103-ccd8-4a7b-9a4f-d94eef0dde24 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:49.795249 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec39d092-b9e5-467c-97d3-549e11fa522d None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:49.809686 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:49.826966 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee1c103-ccd8-4a7b-9a4f-d94eef0dde24 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:49.833841 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c349a8-7a01-4e2b-a0de-8a3a6d58c806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:49.834480 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c349a8-7a01-4e2b-a0de-8a3a6d58c806 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:49.834763 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c349a8-7a01-4e2b-a0de-8a3a6d58c806 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:49.835241 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c349a8-7a01-4e2b-a0de-8a3a6d58c806 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:49.846011 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c349a8-7a01-4e2b-a0de-8a3a6d58c806 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:49.847498 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c349a8-7a01-4e2b-a0de-8a3a6d58c806 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:49.862795 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-b4c349a8-7a01-4e2b-a0de-8a3a6d58c806 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:33:49.862795 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:49.862795 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:49.862795 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:49.862795 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:49.862795 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:49.862795 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:49.862795 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:49.862795 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:49.862795 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:49.862795 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:49.862795 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:49.862795 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:33:49.862795 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:33:49.862795 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:33:49.862795 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:33:49.863767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:49.863767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:49.863767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:49.863767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:49.863767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:49.863767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:49.863767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:49.863767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:49.863767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:49.863767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:49.863767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:49.863767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:49.863767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:49.863767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:49.863767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:49.863767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:49.863767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:33:49.865393 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:33:49.865393 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:33:49.865393 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:33:49.865393 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 263/525] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:33:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:49.865393 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c349a8-7a01-4e2b-a0de-8a3a6d58c806 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:49.871000 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee41449-022b-4284-a365-39d5a5b622e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:49.871653 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee41449-022b-4284-a365-39d5a5b622e6 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:49.871936 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee41449-022b-4284-a365-39d5a5b622e6 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:49.872207 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee41449-022b-4284-a365-39d5a5b622e6 None admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:49.872855 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee41449-022b-4284-a365-39d5a5b622e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:49.873157 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee41449-022b-4284-a365-39d5a5b622e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:49.874534 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee41449-022b-4284-a365-39d5a5b622e6 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:49.887311 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 263/526] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:33:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:49.887311 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee41449-022b-4284-a365-39d5a5b622e6 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:49.951267 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:49.975680 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c349a8-7a01-4e2b-a0de-8a3a6d58c806 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:50.005768 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee41449-022b-4284-a365-39d5a5b622e6 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:50.011742 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb926df3-89c1-47f9-b3be-21582a881274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:50.012381 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb926df3-89c1-47f9-b3be-21582a881274 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:50.012667 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb926df3-89c1-47f9-b3be-21582a881274 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:50.012941 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb926df3-89c1-47f9-b3be-21582a881274 None admin] PATH_INFO: `/v3/projects/service` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:50.019842 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb926df3-89c1-47f9-b3be-21582a881274 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:50.021302 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb926df3-89c1-47f9-b3be-21582a881274 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:50.034760 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-bb926df3-89c1-47f9-b3be-21582a881274 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:33:50.034760 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:50.034760 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:50.034760 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:50.034760 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:50.034760 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:50.034760 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:50.034760 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:50.034760 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:50.034760 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:50.034760 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:50.034760 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:50.034760 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 10:33:50.034760 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 10:33:50.034760 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 10:33:50.036464 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 10:33:50.036464 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:50.036464 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:50.036464 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:50.036464 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:50.036464 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:50.036464 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:50.036464 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:50.036464 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:50.036464 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:50.036464 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:50.036464 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:50.036464 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:50.036464 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:50.036464 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:50.036464 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:50.036464 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:50.038316 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 10:33:50.038316 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 10:33:50.038316 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 10:33:50.038316 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 10:33:50.038316 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 10:33:50.038316 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 10:33:50.038316 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:33:50.038316 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:33:50.038316 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 264/527] 10.222.0.15 () {56 vars in 1111 bytes} [Sun Sep 11 10:33:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:50.039760 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb926df3-89c1-47f9-b3be-21582a881274 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:50.053016 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47defea-1be5-4112-8ecd-2fdfcd4cc3eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:50.053582 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47defea-1be5-4112-8ecd-2fdfcd4cc3eb None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:50.053844 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47defea-1be5-4112-8ecd-2fdfcd4cc3eb None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:50.054666 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47defea-1be5-4112-8ecd-2fdfcd4cc3eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:50.065874 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47defea-1be5-4112-8ecd-2fdfcd4cc3eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:50.067046 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:50.067336 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47defea-1be5-4112-8ecd-2fdfcd4cc3eb None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:50.083698 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-a47defea-1be5-4112-8ecd-2fdfcd4cc3eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:33:50.083698 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:50.083698 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:50.083698 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:50.083698 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:50.083698 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:50.083698 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:50.083698 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:50.083698 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:50.083698 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:50.083698 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:50.083698 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:50.083698 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:33:50.083698 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:33:50.083698 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:33:50.083698 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:33:50.084781 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:50.084781 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:50.084781 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:50.084781 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:50.084781 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:50.084781 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:50.084781 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:50.084781 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:50.084781 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:50.084781 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:50.084781 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:50.084781 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:50.084781 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:50.084781 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:50.084781 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:50.084781 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:50.084781 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:33:50.087088 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:33:50.087088 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:33:50.087088 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:33:50.089889 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 264/528] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:33:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:50.090741 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47defea-1be5-4112-8ecd-2fdfcd4cc3eb None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:50.150314 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb926df3-89c1-47f9-b3be-21582a881274 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:50.184047 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df27ab19-61f1-48ea-a5d1-106928fc454a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:50.184657 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df27ab19-61f1-48ea-a5d1-106928fc454a None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:50.184940 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df27ab19-61f1-48ea-a5d1-106928fc454a None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:50.185213 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df27ab19-61f1-48ea-a5d1-106928fc454a None admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:50.186177 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df27ab19-61f1-48ea-a5d1-106928fc454a None admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:50.186177 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df27ab19-61f1-48ea-a5d1-106928fc454a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:50.187548 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df27ab19-61f1-48ea-a5d1-106928fc454a None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:50.204207 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 265/529] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:33:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:50.204382 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47defea-1be5-4112-8ecd-2fdfcd4cc3eb None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:50.204516 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df27ab19-61f1-48ea-a5d1-106928fc454a None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:50.214731 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:50.244289 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95f24da-e90d-4d8e-a1d2-10994141c9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:50.244960 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f24da-e90d-4d8e-a1d2-10994141c9ce None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:50.245245 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f24da-e90d-4d8e-a1d2-10994141c9ce None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:50.245518 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f24da-e90d-4d8e-a1d2-10994141c9ce None admin] PATH_INFO: `/v3/projects/service` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:50.253594 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95f24da-e90d-4d8e-a1d2-10994141c9ce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:50.255038 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95f24da-e90d-4d8e-a1d2-10994141c9ce None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:50.267402 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-c95f24da-e90d-4d8e-a1d2-10994141c9ce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:33:50.267402 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:50.267402 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:50.267402 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:50.267402 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:50.267402 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:50.267402 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:50.267402 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:50.267402 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:50.267402 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:50.267402 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:50.267402 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:50.267402 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 10:33:50.267402 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 10:33:50.267402 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 10:33:50.268290 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 10:33:50.268290 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:50.268290 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:50.268290 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:50.268290 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:50.268290 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:50.268290 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:50.268290 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:50.268290 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:50.268290 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:50.268290 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:50.268290 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:50.268290 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:50.268290 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:50.268290 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:50.268290 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:50.268290 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:50.269149 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 10:33:50.269149 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 10:33:50.269149 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 10:33:50.269149 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 10:33:50.269149 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 10:33:50.269149 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 10:33:50.269149 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:33:50.269149 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:33:50.269149 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 265/530] 10.222.0.15 () {56 vars in 1111 bytes} [Sun Sep 11 10:33:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:50.269632 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95f24da-e90d-4d8e-a1d2-10994141c9ce None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:50.305736 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:50.325963 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df27ab19-61f1-48ea-a5d1-106928fc454a None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:50.361304 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b901e51-f04c-4666-95ac-382698a5a1e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:50.361916 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b901e51-f04c-4666-95ac-382698a5a1e3 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:50.362194 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b901e51-f04c-4666-95ac-382698a5a1e3 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:50.362468 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b901e51-f04c-4666-95ac-382698a5a1e3 None admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:50.363151 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b901e51-f04c-4666-95ac-382698a5a1e3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:50.363478 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b901e51-f04c-4666-95ac-382698a5a1e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:50.364901 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b901e51-f04c-4666-95ac-382698a5a1e3 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:50.380399 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95f24da-e90d-4d8e-a1d2-10994141c9ce None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:50.384114 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 266/531] 10.222.0.15 () {56 vars in 1148 bytes} [Sun Sep 11 10:33:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:50.385092 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b901e51-f04c-4666-95ac-382698a5a1e3 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:50.423268 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8462c26f-4ae2-48cd-8d2f-929ed2e1aa70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:50.423912 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8462c26f-4ae2-48cd-8d2f-929ed2e1aa70 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:50.424197 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8462c26f-4ae2-48cd-8d2f-929ed2e1aa70 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:50.424885 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8462c26f-4ae2-48cd-8d2f-929ed2e1aa70 None admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:50.425534 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8462c26f-4ae2-48cd-8d2f-929ed2e1aa70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:50.425842 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8462c26f-4ae2-48cd-8d2f-929ed2e1aa70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:50.427611 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8462c26f-4ae2-48cd-8d2f-929ed2e1aa70 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:50.440549 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 266/532] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:33:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:50.442097 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8462c26f-4ae2-48cd-8d2f-929ed2e1aa70 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:50.476237 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:50.494655 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b901e51-f04c-4666-95ac-382698a5a1e3 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:50.530398 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4796677a-2ba8-4ee2-972e-83f4dc4129a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:50.531158 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4796677a-2ba8-4ee2-972e-83f4dc4129a6 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:50.531528 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4796677a-2ba8-4ee2-972e-83f4dc4129a6 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:50.531874 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4796677a-2ba8-4ee2-972e-83f4dc4129a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:50.538772 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4796677a-2ba8-4ee2-972e-83f4dc4129a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:50.540262 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4796677a-2ba8-4ee2-972e-83f4dc4129a6 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:50.551456 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-4796677a-2ba8-4ee2-972e-83f4dc4129a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:33:50.551456 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:50.551456 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:50.551456 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:50.551456 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:50.551456 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:50.551456 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:50.551456 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:50.551456 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:50.551456 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:50.551456 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:50.551456 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:50.551456 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 10:33:50.551456 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 10:33:50.551456 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 10:33:50.552490 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 10:33:50.552490 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:50.552490 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:50.552490 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:50.552490 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:50.552490 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:50.552490 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:50.552490 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:50.552490 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:50.552490 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:50.552490 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:50.552490 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:50.552490 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:50.552490 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:50.552490 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:50.552490 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:50.552490 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:50.553697 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 10:33:50.553697 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 10:33:50.553697 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 10:33:50.553697 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 10:33:50.553697 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 10:33:50.553697 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 10:33:50.553697 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:33:50.553697 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:33:50.553697 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 267/533] 10.222.0.15 () {56 vars in 1111 bytes} [Sun Sep 11 10:33:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:50.553697 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4796677a-2ba8-4ee2-972e-83f4dc4129a6 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:50.558009 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:50.560411 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8462c26f-4ae2-48cd-8d2f-929ed2e1aa70 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:50.598248 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6c1de0-e93f-4b26-ba7b-09cc42a0d402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:50.598864 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c1de0-e93f-4b26-ba7b-09cc42a0d402 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:50.599452 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c1de0-e93f-4b26-ba7b-09cc42a0d402 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:50.599452 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c1de0-e93f-4b26-ba7b-09cc42a0d402 None admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:50.600131 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6c1de0-e93f-4b26-ba7b-09cc42a0d402 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:50.600438 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6c1de0-e93f-4b26-ba7b-09cc42a0d402 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:50.602383 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6c1de0-e93f-4b26-ba7b-09cc42a0d402 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:50.622225 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 267/534] 10.222.0.15 () {56 vars in 1148 bytes} [Sun Sep 11 10:33:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:50.624619 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6c1de0-e93f-4b26-ba7b-09cc42a0d402 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:50.661782 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4796677a-2ba8-4ee2-972e-83f4dc4129a6 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:50.704726 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c32e6a-88f4-40ac-9a3a-9bd1d81c791f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:50.705331 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c32e6a-88f4-40ac-9a3a-9bd1d81c791f None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:50.706038 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c32e6a-88f4-40ac-9a3a-9bd1d81c791f None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:50.706315 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c32e6a-88f4-40ac-9a3a-9bd1d81c791f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:50.707033 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c32e6a-88f4-40ac-9a3a-9bd1d81c791f None admin] RBAC: Adding query filter params (role.id=3cb7ebbc707d453586b3d0f4ab3cf103, scope.project.id=5e1f845f99f04202b83621cb65e0adc2, user.id=1632da5fbe384b5c849be1d4d86c0f1d) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:50.707340 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c32e6a-88f4-40ac-9a3a-9bd1d81c791f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:50.709467 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c32e6a-88f4-40ac-9a3a-9bd1d81c791f None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:50.716281 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 268/535] 10.222.0.15 () {56 vars in 1374 bytes} [Sun Sep 11 10:33:50 2022] GET /identity/v3/role_assignments?user.id=1632da5fbe384b5c849be1d4d86c0f1d&scope.project.id=5e1f845f99f04202b83621cb65e0adc2&role.id=3cb7ebbc707d453586b3d0f4ab3cf103 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:50.717527 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c32e6a-88f4-40ac-9a3a-9bd1d81c791f None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:50.724106 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:50.740914 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6c1de0-e93f-4b26-ba7b-09cc42a0d402 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:50.791741 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62bceac-d4a9-4117-9078-579879a6303b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:50.792449 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62bceac-d4a9-4117-9078-579879a6303b None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:50.792735 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62bceac-d4a9-4117-9078-579879a6303b None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:50.793007 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62bceac-d4a9-4117-9078-579879a6303b None admin] PATH_INFO: `/v3/projects/service` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:50.802417 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62bceac-d4a9-4117-9078-579879a6303b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:50.805258 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62bceac-d4a9-4117-9078-579879a6303b None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:50.812606 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:50.821888 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-c62bceac-d4a9-4117-9078-579879a6303b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:33:50.821888 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:50.821888 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:50.821888 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:50.821888 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:50.821888 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:50.821888 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:50.821888 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:50.821888 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:50.821888 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:50.821888 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:50.821888 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:50.821888 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 10:33:50.821888 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 10:33:50.821888 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 10:33:50.823035 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 10:33:50.823035 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:50.823035 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:50.823035 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:50.823035 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:50.823035 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:50.823035 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:50.823035 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:50.823035 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:50.823035 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:50.823035 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:50.823035 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:50.823035 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:50.823035 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:50.823035 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:50.823035 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:50.823035 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:50.824041 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 10:33:50.824041 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 10:33:50.824041 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 10:33:50.824041 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 10:33:50.824041 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 10:33:50.824041 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 10:33:50.824041 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:33:50.824041 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:33:50.824041 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 268/536] 10.222.0.15 () {56 vars in 1111 bytes} [Sun Sep 11 10:33:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:50.828877 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62bceac-d4a9-4117-9078-579879a6303b None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:50.859759 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c32e6a-88f4-40ac-9a3a-9bd1d81c791f None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:50.902598 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ec222-00e1-4871-bf83-af6d316d6e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:50.903226 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ec222-00e1-4871-bf83-af6d316d6e81 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:50.903535 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ec222-00e1-4871-bf83-af6d316d6e81 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:50.903815 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ec222-00e1-4871-bf83-af6d316d6e81 None admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:50.905746 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77ec222-00e1-4871-bf83-af6d316d6e81 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:50.905746 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77ec222-00e1-4871-bf83-af6d316d6e81 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:50.906228 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77ec222-00e1-4871-bf83-af6d316d6e81 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:50.920403 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 269/537] 10.222.0.15 () {56 vars in 1148 bytes} [Sun Sep 11 10:33:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:50.921331 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ec222-00e1-4871-bf83-af6d316d6e81 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:50.953146 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62bceac-d4a9-4117-9078-579879a6303b None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:50.993531 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d76071-090b-4c57-8adb-627a8b5c25b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:50.994283 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d76071-090b-4c57-8adb-627a8b5c25b9 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:50.994811 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d76071-090b-4c57-8adb-627a8b5c25b9 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:50.995187 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d76071-090b-4c57-8adb-627a8b5c25b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:50.996041 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d76071-090b-4c57-8adb-627a8b5c25b9 None admin] RBAC: Adding query filter params (role.id=3cb7ebbc707d453586b3d0f4ab3cf103, scope.project.id=5e1f845f99f04202b83621cb65e0adc2, user.id=2bd5356829b6454993bab9bad2f79fed) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:50.996447 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d76071-090b-4c57-8adb-627a8b5c25b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:50.997987 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d76071-090b-4c57-8adb-627a8b5c25b9 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:51.005305 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 269/538] 10.222.0.15 () {56 vars in 1374 bytes} [Sun Sep 11 10:33:50 2022] GET /identity/v3/role_assignments?user.id=2bd5356829b6454993bab9bad2f79fed&scope.project.id=5e1f845f99f04202b83621cb65e0adc2&role.id=3cb7ebbc707d453586b3d0f4ab3cf103 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:51.006258 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d76071-090b-4c57-8adb-627a8b5c25b9 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:51.047857 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:51.083483 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ec222-00e1-4871-bf83-af6d316d6e81 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:51.144244 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:51.164879 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dff3c9-1823-4366-bdcd-dad5c9893153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:51.165524 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dff3c9-1823-4366-bdcd-dad5c9893153 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:51.165808 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dff3c9-1823-4366-bdcd-dad5c9893153 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:51.166087 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dff3c9-1823-4366-bdcd-dad5c9893153 None admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:51.166790 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dff3c9-1823-4366-bdcd-dad5c9893153 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:51.167375 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dff3c9-1823-4366-bdcd-dad5c9893153 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:51.168593 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dff3c9-1823-4366-bdcd-dad5c9893153 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:51.192333 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d76071-090b-4c57-8adb-627a8b5c25b9 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:51.199119 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 270/539] 10.222.0.15 () {56 vars in 1148 bytes} [Sun Sep 11 10:33:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:51.204615 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dff3c9-1823-4366-bdcd-dad5c9893153 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:51.238806 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caacf93-ecb4-4dd0-8828-b4eb1a5b3eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:51.239386 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caacf93-ecb4-4dd0-8828-b4eb1a5b3eac None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:51.239691 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caacf93-ecb4-4dd0-8828-b4eb1a5b3eac None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:51.239957 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caacf93-ecb4-4dd0-8828-b4eb1a5b3eac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:51.240677 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caacf93-ecb4-4dd0-8828-b4eb1a5b3eac None admin] RBAC: Adding query filter params (role.id=3cb7ebbc707d453586b3d0f4ab3cf103, scope.project.id=5e1f845f99f04202b83621cb65e0adc2, user.id=0bbb481abb8e43788e3412ac7d908ad3) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:51.240979 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caacf93-ecb4-4dd0-8828-b4eb1a5b3eac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:51.242548 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caacf93-ecb4-4dd0-8828-b4eb1a5b3eac None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:51.248828 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 270/540] 10.222.0.15 () {56 vars in 1374 bytes} [Sun Sep 11 10:33:51 2022] GET /identity/v3/role_assignments?user.id=0bbb481abb8e43788e3412ac7d908ad3&scope.project.id=5e1f845f99f04202b83621cb65e0adc2&role.id=3cb7ebbc707d453586b3d0f4ab3cf103 => generated 259 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:51.390589 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dff3c9-1823-4366-bdcd-dad5c9893153 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:51.441832 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7f8b88-bd68-409d-9dc0-d8990f999d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:51.442724 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f8b88-bd68-409d-9dc0-d8990f999d0b None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:51.443151 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f8b88-bd68-409d-9dc0-d8990f999d0b None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:51.443599 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f8b88-bd68-409d-9dc0-d8990f999d0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:51.444523 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7f8b88-bd68-409d-9dc0-d8990f999d0b None admin] RBAC: Adding query filter params (role.id=3cb7ebbc707d453586b3d0f4ab3cf103, scope.project.id=5e1f845f99f04202b83621cb65e0adc2, user.id=40072bc8dcb5454382a93e20b63f8594) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:51.445124 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7f8b88-bd68-409d-9dc0-d8990f999d0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:51.446800 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7f8b88-bd68-409d-9dc0-d8990f999d0b None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:51.453639 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 271/541] 10.222.0.15 () {56 vars in 1374 bytes} [Sun Sep 11 10:33:51 2022] GET /identity/v3/role_assignments?user.id=40072bc8dcb5454382a93e20b63f8594&scope.project.id=5e1f845f99f04202b83621cb65e0adc2&role.id=3cb7ebbc707d453586b3d0f4ab3cf103 => generated 259 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:33:51.576483 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c879f-d577-40bc-8971-d203dc6503d7 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:51.576951 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c879f-d577-40bc-8971-d203dc6503d7 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:51.577379 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c879f-d577-40bc-8971-d203dc6503d7 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:51.579105 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 271/542] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:33:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:33:51.584814 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64e5d5-dab7-4643-913f-08cb5e52f21a None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:51.585103 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64e5d5-dab7-4643-913f-08cb5e52f21a None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:51.585935 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64e5d5-dab7-4643-913f-08cb5e52f21a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:51.609111 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-0a64e5d5-dab7-4643-913f-08cb5e52f21a None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:33:51.610380 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-0a64e5d5-dab7-4643-913f-08cb5e52f21a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:33:51.679551 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:51.700796 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 272/543] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:33:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:33:52.099145 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08c879f-d577-40bc-8971-d203dc6503d7 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:52.204718 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:52.240092 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08c879f-d577-40bc-8971-d203dc6503d7 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:52.286433 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce96b07f-9e6e-4be2-a406-34fe47a3087f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:52.287080 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce96b07f-9e6e-4be2-a406-34fe47a3087f None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:52.287362 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce96b07f-9e6e-4be2-a406-34fe47a3087f None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:52.287667 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce96b07f-9e6e-4be2-a406-34fe47a3087f None admin] PATH_INFO: `/v3/roles/service` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:52.299052 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce96b07f-9e6e-4be2-a406-34fe47a3087f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:33:52.305247 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-ce96b07f-9e6e-4be2-a406-34fe47a3087f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 10:33:52.305247 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:52.305247 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:52.305247 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:52.305247 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:52.305247 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:52.305247 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:52.305247 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:52.305247 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:52.305247 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:52.305247 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:52.305247 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:52.305247 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:33:52.305247 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:33:52.305247 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:33:52.305247 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:33:52.306159 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:33:52.306159 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:33:52.306159 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:33:52.306159 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:33:52.306159 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:52.306159 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:52.306159 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:52.306159 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:52.306159 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:52.306159 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:52.306159 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:52.306159 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:52.306159 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:52.306159 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:52.306159 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:52.306159 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:52.306962 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:52.306962 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:52.306962 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:52.306962 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:52.306962 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:33:52.306962 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:33:52.306962 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:33:52.306962 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:33:52.306962 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:33:52.306962 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:33:52.306962 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 10:33:52.306962 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:33:52.311921 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 272/544] 10.222.0.15 () {56 vars in 1102 bytes} [Sun Sep 11 10:33:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:52.317867 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a64e5d5-dab7-4643-913f-08cb5e52f21a None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:52.438918 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a64e5d5-dab7-4643-913f-08cb5e52f21a None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:52.486192 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1f706c-0c02-4f95-9728-4e982da9e9b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:52.486826 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1f706c-0c02-4f95-9728-4e982da9e9b3 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:52.487114 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1f706c-0c02-4f95-9728-4e982da9e9b3 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:52.487391 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1f706c-0c02-4f95-9728-4e982da9e9b3 None admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:52.488105 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1f706c-0c02-4f95-9728-4e982da9e9b3 None admin] RBAC: Adding query filter params (name=service) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:52.488406 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1f706c-0c02-4f95-9728-4e982da9e9b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:52.489815 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1f706c-0c02-4f95-9728-4e982da9e9b3 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:52.499083 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 273/545] 10.222.0.15 () {56 vars in 1103 bytes} [Sun Sep 11 10:33:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:52.504881 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce96b07f-9e6e-4be2-a406-34fe47a3087f None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:52.639880 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce96b07f-9e6e-4be2-a406-34fe47a3087f None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:52.689113 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d685da-a033-41ab-be23-041219536ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:52.689671 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d685da-a033-41ab-be23-041219536ace None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:52.689947 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d685da-a033-41ab-be23-041219536ace None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:52.690215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d685da-a033-41ab-be23-041219536ace None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:52.693083 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:52.707022 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d685da-a033-41ab-be23-041219536ace None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:52.708508 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d685da-a033-41ab-be23-041219536ace None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:52.728282 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-48d685da-a033-41ab-be23-041219536ace None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:33:52.728282 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:52.728282 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:52.728282 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:52.728282 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:52.728282 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:52.728282 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:52.728282 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:52.728282 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:52.728282 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:52.728282 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:52.728282 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:52.728282 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:33:52.728282 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:33:52.728282 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:33:52.728282 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:33:52.729264 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:52.729264 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:52.729264 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:52.729264 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:52.729264 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:52.729264 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:52.729264 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:52.729264 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:52.729264 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:52.729264 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:52.729264 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:52.729264 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:52.729264 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:52.729264 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:52.729264 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:52.729264 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:52.729264 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:33:52.730738 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:33:52.730738 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:33:52.730738 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:33:52.730738 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 273/546] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:33:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:52.735075 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1f706c-0c02-4f95-9728-4e982da9e9b3 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:52.817457 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:52.902171 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1f706c-0c02-4f95-9728-4e982da9e9b3 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:52.949480 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ed6468-13e7-470f-b948-0a0683aa29d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:52.950060 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed6468-13e7-470f-b948-0a0683aa29d2 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:52.950343 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed6468-13e7-470f-b948-0a0683aa29d2 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:52.950615 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed6468-13e7-470f-b948-0a0683aa29d2 None admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:52.951254 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ed6468-13e7-470f-b948-0a0683aa29d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:52.951581 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ed6468-13e7-470f-b948-0a0683aa29d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:52.952992 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ed6468-13e7-470f-b948-0a0683aa29d2 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:52.969936 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 274/547] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:33:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:52.977318 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d685da-a033-41ab-be23-041219536ace None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:53.172834 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d685da-a033-41ab-be23-041219536ace None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:53.225103 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b0482f-b0af-46c1-b485-54078699c79d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:53.225653 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b0482f-b0af-46c1-b485-54078699c79d None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:53.226579 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b0482f-b0af-46c1-b485-54078699c79d None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:53.227234 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b0482f-b0af-46c1-b485-54078699c79d None admin] PATH_INFO: `/v3/users/swift` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:53.237374 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b0482f-b0af-46c1-b485-54078699c79d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:53.238812 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b0482f-b0af-46c1-b485-54078699c79d None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:53.255058 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-f9b0482f-b0af-46c1-b485-54078699c79d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 11 10:33:53.255058 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:53.255058 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:53.255058 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:53.255058 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:53.255058 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:53.255058 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:53.255058 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:53.255058 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:53.255058 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:53.255058 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:53.255058 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:53.255058 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 10:33:53.255058 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 10:33:53.255058 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 10:33:53.255058 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 10:33:53.256940 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:53.256940 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:53.256940 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 10:33:53.256940 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:33:53.256940 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 10:33:53.256940 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:33:53.256940 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:53.256940 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:53.256940 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:53.256940 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:53.256940 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:53.256940 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:53.256940 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:53.256940 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:53.256940 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:53.256940 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:53.256940 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:53.256940 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:53.258355 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:53.258355 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:53.258355 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 10:33:53.258355 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 10:33:53.258355 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 10:33:53.258355 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 10:33:53.258355 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 10:33:53.258355 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 10:33:53.258355 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 11 10:33:53.258355 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:33:53.258355 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 274/548] 10.222.0.15 () {56 vars in 1096 bytes} [Sun Sep 11 10:33:52 2022] GET /identity/v3/users/swift => generated 83 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:53.262402 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ed6468-13e7-470f-b948-0a0683aa29d2 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:53.283703 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da808443-a3e2-4c16-893b-b6165d8a58fa None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:53.284004 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da808443-a3e2-4c16-893b-b6165d8a58fa None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:53.284247 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da808443-a3e2-4c16-893b-b6165d8a58fa None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:53.287165 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 275/549] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:33:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 10:33:53.292783 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0017b795-9572-48ae-896f-7b47288d8f0d None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:53.293371 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0017b795-9572-48ae-896f-7b47288d8f0d None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:53.293851 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0017b795-9572-48ae-896f-7b47288d8f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:53.313831 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-0017b795-9572-48ae-896f-7b47288d8f0d None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:33:53.315360 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-0017b795-9572-48ae-896f-7b47288d8f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:33:53.364140 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:53.400607 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 276/550] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:33:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:33:53.439663 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ed6468-13e7-470f-b948-0a0683aa29d2 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:53.443636 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:53.488226 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bc12ef-5a0c-4824-832d-b1886939e91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:53.489190 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bc12ef-5a0c-4824-832d-b1886939e91b None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:53.489670 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bc12ef-5a0c-4824-832d-b1886939e91b None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:53.490157 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bc12ef-5a0c-4824-832d-b1886939e91b None admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:53.491078 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bc12ef-5a0c-4824-832d-b1886939e91b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:53.491673 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bc12ef-5a0c-4824-832d-b1886939e91b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:53.493323 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bc12ef-5a0c-4824-832d-b1886939e91b None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:53.514634 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 275/551] 10.222.0.15 () {56 vars in 1135 bytes} [Sun Sep 11 10:33:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:53.520472 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0017b795-9572-48ae-896f-7b47288d8f0d None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:53.635499 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0017b795-9572-48ae-896f-7b47288d8f0d None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:53.676015 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8d3319-588e-40a1-84df-983686186a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:53.677034 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8d3319-588e-40a1-84df-983686186a30 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:53.677538 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8d3319-588e-40a1-84df-983686186a30 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:53.678031 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8d3319-588e-40a1-84df-983686186a30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:53.731808 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:53.760366 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8d3319-588e-40a1-84df-983686186a30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:53.762244 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8d3319-588e-40a1-84df-983686186a30 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:53.865411 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 277/552] 10.222.0.15 () {58 vars in 1282 bytes} [Sun Sep 11 10:33:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 11 10:33:53.871741 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bc12ef-5a0c-4824-832d-b1886939e91b None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:53.885711 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8d3319-588e-40a1-84df-983686186a30 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:53.992741 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:54.028546 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8d3319-588e-40a1-84df-983686186a30 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:54.030942 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bc12ef-5a0c-4824-832d-b1886939e91b None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:54.072318 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89cf062-e924-46ae-aaed-65eeebc8a5f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:54.072965 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89cf062-e924-46ae-aaed-65eeebc8a5f6 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:54.073249 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89cf062-e924-46ae-aaed-65eeebc8a5f6 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:54.073526 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89cf062-e924-46ae-aaed-65eeebc8a5f6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:54.077367 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fecee02-b877-4530-876a-94bc5785dbc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:54.078013 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fecee02-b877-4530-876a-94bc5785dbc8 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:54.078292 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fecee02-b877-4530-876a-94bc5785dbc8 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:54.078560 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fecee02-b877-4530-876a-94bc5785dbc8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:54.086079 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89cf062-e924-46ae-aaed-65eeebc8a5f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:33:54.091481 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fecee02-b877-4530-876a-94bc5785dbc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:54.092297 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-e89cf062-e924-46ae-aaed-65eeebc8a5f6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 10:33:54.092297 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:54.092297 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:54.092297 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:54.092297 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:54.092297 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:54.092297 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:54.092297 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:54.092297 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:54.092297 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:54.092297 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:54.092297 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:54.092297 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:33:54.092297 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:33:54.092297 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:33:54.092297 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:33:54.093212 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:33:54.093212 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:33:54.093212 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:33:54.093212 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:33:54.093212 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:54.093212 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:54.093212 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:54.093212 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:54.093212 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:54.093212 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:54.093212 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:54.093212 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:54.093212 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:54.093212 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:54.093212 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:54.093212 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:54.094423 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:54.094423 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:54.094423 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:54.094423 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:54.094423 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:33:54.094423 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:33:54.094423 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:33:54.094423 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:33:54.094423 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:33:54.094423 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:33:54.094423 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 10:33:54.094423 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:33:54.095060 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fecee02-b877-4530-876a-94bc5785dbc8 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:54.095139 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 278/553] 10.222.0.15 () {56 vars in 1102 bytes} [Sun Sep 11 10:33:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:54.095139 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e05023-d8d8-4723-9b20-effb93204000 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:54.095270 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e05023-d8d8-4723-9b20-effb93204000 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:54.095622 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e05023-d8d8-4723-9b20-effb93204000 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:54.100356 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 279/554] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:33:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:33:54.103652 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e05023-d8d8-4723-9b20-effb93204000 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:54.116261 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-4fecee02-b877-4530-876a-94bc5785dbc8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:33:54.116261 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:54.116261 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:54.116261 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:54.116261 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:54.116261 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:54.116261 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:54.116261 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:54.116261 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:54.116261 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:54.116261 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:54.116261 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:54.116261 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:33:54.116261 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:33:54.116261 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:33:54.116261 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:33:54.117456 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:54.117456 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:54.117456 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:54.117456 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:54.117456 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:54.117456 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:54.117456 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:54.117456 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:54.117456 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:54.117456 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:54.117456 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:54.117456 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:54.117456 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:54.117456 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:54.117456 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:54.117456 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:54.117456 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:33:54.118324 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:33:54.118324 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:33:54.118324 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:33:54.118324 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 276/555] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:33:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:54.119698 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85fac09-b30c-4748-8930-3c5b592ec6a7 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:54.120249 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85fac09-b30c-4748-8930-3c5b592ec6a7 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:54.120525 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85fac09-b30c-4748-8930-3c5b592ec6a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:54.138805 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-a85fac09-b30c-4748-8930-3c5b592ec6a7 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:33:54.140200 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-a85fac09-b30c-4748-8930-3c5b592ec6a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:33:54.150100 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:54.206576 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 277/556] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:33:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:33:54.207905 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f524216-0f42-4ac6-859b-bc16bb981371 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:54.208198 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f524216-0f42-4ac6-859b-bc16bb981371 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:54.208472 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f524216-0f42-4ac6-859b-bc16bb981371 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:54.211611 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 278/557] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:33:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:33:54.211763 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e05023-d8d8-4723-9b20-effb93204000 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:54.212181 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5480e0-a93f-4670-8739-ad164dbee7ef None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:54.212181 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5480e0-a93f-4670-8739-ad164dbee7ef None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:54.212181 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5480e0-a93f-4670-8739-ad164dbee7ef None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:54.217399 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 279/558] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:33:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:33:54.217399 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5480e0-a93f-4670-8739-ad164dbee7ef None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:54.249675 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb4a8c1-db62-4449-8412-42896272c932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:54.250280 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb4a8c1-db62-4449-8412-42896272c932 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:54.250560 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb4a8c1-db62-4449-8412-42896272c932 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:54.250830 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb4a8c1-db62-4449-8412-42896272c932 None admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:54.251520 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb4a8c1-db62-4449-8412-42896272c932 None admin] RBAC: Adding query filter params (name=service) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:54.251815 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb4a8c1-db62-4449-8412-42896272c932 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:54.253182 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb4a8c1-db62-4449-8412-42896272c932 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:54.265403 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 280/559] 10.222.0.15 () {56 vars in 1103 bytes} [Sun Sep 11 10:33:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:54.265403 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fbd950-ddaf-4944-babd-207acd90f7c1 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:54.265403 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fbd950-ddaf-4944-babd-207acd90f7c1 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:54.265403 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fbd950-ddaf-4944-babd-207acd90f7c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:54.265878 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:54.282871 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-95fbd950-ddaf-4944-babd-207acd90f7c1 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:33:54.284005 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-95fbd950-ddaf-4944-babd-207acd90f7c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:33:54.343574 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5480e0-a93f-4670-8739-ad164dbee7ef None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:54.359558 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 281/560] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:33:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:33:54.360939 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b30c8e5-6706-4db8-a023-d12180ecab45 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:54.361297 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b30c8e5-6706-4db8-a023-d12180ecab45 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:54.361640 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b30c8e5-6706-4db8-a023-d12180ecab45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:54.382595 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-1b30c8e5-6706-4db8-a023-d12180ecab45 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:33:54.384053 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-1b30c8e5-6706-4db8-a023-d12180ecab45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:33:54.389292 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5312d672-01c8-49d7-a67e-c272b3b9d8ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:54.390153 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5312d672-01c8-49d7-a67e-c272b3b9d8ad None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:54.390632 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5312d672-01c8-49d7-a67e-c272b3b9d8ad None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:54.391086 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5312d672-01c8-49d7-a67e-c272b3b9d8ad None admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:54.391927 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5312d672-01c8-49d7-a67e-c272b3b9d8ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:54.392493 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5312d672-01c8-49d7-a67e-c272b3b9d8ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:54.394058 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5312d672-01c8-49d7-a67e-c272b3b9d8ad None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:54.411085 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 280/561] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:33:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:54.412265 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5312d672-01c8-49d7-a67e-c272b3b9d8ad None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:54.417424 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:54.459662 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 282/562] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:33:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:33:54.461363 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b30c8e5-6706-4db8-a023-d12180ecab45 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:54.537380 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:54.540298 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5312d672-01c8-49d7-a67e-c272b3b9d8ad None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:54.574843 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb4e6de-7678-4ce2-8553-e8fc970eb416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:54.575641 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb4e6de-7678-4ce2-8553-e8fc970eb416 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:54.576015 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb4e6de-7678-4ce2-8553-e8fc970eb416 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:54.576418 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb4e6de-7678-4ce2-8553-e8fc970eb416 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:54.589985 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb4e6de-7678-4ce2-8553-e8fc970eb416 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:54.592142 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb4e6de-7678-4ce2-8553-e8fc970eb416 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:54.593541 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b30c8e5-6706-4db8-a023-d12180ecab45 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:54.610757 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-feb4e6de-7678-4ce2-8553-e8fc970eb416 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:33:54.610757 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:54.610757 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:54.610757 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:54.610757 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:54.610757 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:54.610757 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:54.610757 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:54.610757 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:54.610757 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:54.610757 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:54.610757 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:54.610757 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:33:54.610757 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:33:54.610757 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:33:54.610757 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:33:54.611887 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:54.611887 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:54.611887 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:54.611887 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:54.611887 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:54.611887 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:54.611887 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:54.611887 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:54.611887 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:54.611887 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:54.611887 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:54.611887 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:54.611887 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:54.611887 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:54.611887 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:54.611887 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:54.611887 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:33:54.612800 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:33:54.612800 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:33:54.612800 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:33:54.612800 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 281/563] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:33:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:54.612800 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb4e6de-7678-4ce2-8553-e8fc970eb416 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:54.635551 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed95f34f-0771-4623-8789-26a85e3cd776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:54.636193 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95f34f-0771-4623-8789-26a85e3cd776 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:54.636483 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95f34f-0771-4623-8789-26a85e3cd776 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:54.636747 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95f34f-0771-4623-8789-26a85e3cd776 None admin] PATH_INFO: `/v3/projects/service` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:54.643987 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed95f34f-0771-4623-8789-26a85e3cd776 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:54.645428 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed95f34f-0771-4623-8789-26a85e3cd776 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:54.658045 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-ed95f34f-0771-4623-8789-26a85e3cd776 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:33:54.658045 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:54.658045 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:54.658045 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:54.658045 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:54.658045 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:54.658045 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:54.658045 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:54.658045 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:54.658045 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:54.658045 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:54.658045 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:54.658045 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 10:33:54.658045 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 10:33:54.658045 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 10:33:54.659042 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 10:33:54.659042 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:54.659042 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:54.659042 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:54.659042 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:54.659042 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:54.659042 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:54.659042 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:54.659042 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:54.659042 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:54.659042 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:54.659042 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:54.659042 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:54.659042 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:54.659042 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:54.659042 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:54.659042 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:54.665864 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 10:33:54.665864 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 10:33:54.665864 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 10:33:54.665864 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 10:33:54.665864 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 10:33:54.665864 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 10:33:54.665864 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:33:54.665864 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:33:54.665864 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 283/564] 10.222.0.15 () {56 vars in 1111 bytes} [Sun Sep 11 10:33:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:54.665864 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed95f34f-0771-4623-8789-26a85e3cd776 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:54.688262 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:54.733115 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb4e6de-7678-4ce2-8553-e8fc970eb416 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:54.766032 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ef4e37-f6dd-442d-9b78-911324a0b389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:54.766657 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef4e37-f6dd-442d-9b78-911324a0b389 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:54.766932 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef4e37-f6dd-442d-9b78-911324a0b389 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:54.767199 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef4e37-f6dd-442d-9b78-911324a0b389 None admin] PATH_INFO: `/v3/roles/service` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:54.771114 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed95f34f-0771-4623-8789-26a85e3cd776 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:54.777863 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ef4e37-f6dd-442d-9b78-911324a0b389 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:33:54.783035 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-a9ef4e37-f6dd-442d-9b78-911324a0b389 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 10:33:54.783035 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:54.783035 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:54.783035 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:54.783035 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:54.783035 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:54.783035 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:54.783035 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:54.783035 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:54.783035 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:54.783035 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:54.783035 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:54.783035 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:33:54.783035 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:33:54.783035 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:33:54.783035 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:33:54.787798 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:33:54.787798 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:33:54.787798 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:33:54.787798 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:33:54.787798 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:54.787798 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:54.787798 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:54.787798 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:54.787798 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:54.787798 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:54.787798 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:54.787798 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:54.787798 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:54.787798 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:54.787798 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:54.787798 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:54.789308 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:54.789308 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:54.789308 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:54.789308 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:54.789308 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:33:54.789308 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:33:54.789308 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:33:54.789308 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:33:54.789308 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:33:54.789308 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:33:54.789308 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 10:33:54.789308 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:33:54.789308 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 282/565] 10.222.0.15 () {56 vars in 1102 bytes} [Sun Sep 11 10:33:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:54.789308 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ef4e37-f6dd-442d-9b78-911324a0b389 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:54.791557 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:54.806879 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00453598-143c-4898-84e5-d850540bfdf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:54.807506 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00453598-143c-4898-84e5-d850540bfdf1 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:54.807790 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00453598-143c-4898-84e5-d850540bfdf1 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:54.808056 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00453598-143c-4898-84e5-d850540bfdf1 None admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:54.808683 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00453598-143c-4898-84e5-d850540bfdf1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:54.808982 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00453598-143c-4898-84e5-d850540bfdf1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:54.810329 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00453598-143c-4898-84e5-d850540bfdf1 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:54.822957 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 284/566] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:33:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:54.823782 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00453598-143c-4898-84e5-d850540bfdf1 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:54.886724 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ef4e37-f6dd-442d-9b78-911324a0b389 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:54.902976 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:54.920562 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1db648-c112-4299-9453-ea6d86b4f601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:54.921184 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1db648-c112-4299-9453-ea6d86b4f601 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:54.921464 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1db648-c112-4299-9453-ea6d86b4f601 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:54.921736 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1db648-c112-4299-9453-ea6d86b4f601 None admin] PATH_INFO: `/v3/roles/service` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:54.922537 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00453598-143c-4898-84e5-d850540bfdf1 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:54.931579 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1db648-c112-4299-9453-ea6d86b4f601 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:33:54.936959 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-ab1db648-c112-4299-9453-ea6d86b4f601 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 10:33:54.936959 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:54.936959 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:54.936959 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:54.936959 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:54.936959 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:54.936959 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:54.936959 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:54.936959 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:54.936959 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:54.936959 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:54.936959 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:54.936959 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:33:54.936959 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:33:54.936959 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:33:54.936959 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:33:54.937919 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:33:54.937919 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:33:54.937919 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:33:54.937919 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:33:54.937919 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:54.937919 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:54.937919 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:54.937919 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:54.937919 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:54.937919 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:54.937919 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:54.937919 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:54.937919 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:54.937919 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:54.937919 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:54.937919 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:54.939241 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:54.939241 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:54.939241 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:54.939241 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:54.939241 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:33:54.939241 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:33:54.939241 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:33:54.939241 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:33:54.939241 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:33:54.939241 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:33:54.939241 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 10:33:54.939241 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:33:54.939241 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 283/567] 10.222.0.15 () {56 vars in 1102 bytes} [Sun Sep 11 10:33:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:54.939241 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1db648-c112-4299-9453-ea6d86b4f601 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:54.958248 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2a7cde-78a2-492e-afa2-8765c4765c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:54.958869 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2a7cde-78a2-492e-afa2-8765c4765c42 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:54.959132 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2a7cde-78a2-492e-afa2-8765c4765c42 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:54.959412 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2a7cde-78a2-492e-afa2-8765c4765c42 None admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:54.960114 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2a7cde-78a2-492e-afa2-8765c4765c42 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:54.961059 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2a7cde-78a2-492e-afa2-8765c4765c42 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:54.962536 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2a7cde-78a2-492e-afa2-8765c4765c42 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:54.973819 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 285/568] 10.222.0.15 () {56 vars in 1148 bytes} [Sun Sep 11 10:33:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:54.974429 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2a7cde-78a2-492e-afa2-8765c4765c42 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:55.016183 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:55.045190 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1db648-c112-4299-9453-ea6d86b4f601 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:55.075105 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2a7cde-78a2-492e-afa2-8765c4765c42 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:55.078163 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497d6b0c-0d17-4bd8-aa10-a5c5ca2c0c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:55.078818 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d6b0c-0d17-4bd8-aa10-a5c5ca2c0c88 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:55.079094 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d6b0c-0d17-4bd8-aa10-a5c5ca2c0c88 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:55.079362 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d6b0c-0d17-4bd8-aa10-a5c5ca2c0c88 None admin] PATH_INFO: `/v3/roles/service` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:55.089905 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497d6b0c-0d17-4bd8-aa10-a5c5ca2c0c88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:33:55.095052 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-497d6b0c-0d17-4bd8-aa10-a5c5ca2c0c88 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 10:33:55.095052 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:55.095052 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:55.095052 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:55.095052 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:55.095052 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:55.095052 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:55.095052 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:55.095052 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:55.095052 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:55.095052 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:55.095052 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:55.095052 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:33:55.095052 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:33:55.095052 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:33:55.095052 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:33:55.099601 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:33:55.099601 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:33:55.099601 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:33:55.099601 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:33:55.099601 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:55.099601 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:55.099601 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:55.099601 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:55.099601 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:55.099601 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:55.099601 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:55.099601 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:55.099601 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:55.099601 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:55.099601 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:55.099601 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:55.100498 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:55.100498 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:55.100498 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:55.100498 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:55.100498 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:33:55.100498 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:33:55.100498 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:33:55.100498 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:33:55.100498 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:33:55.100498 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:33:55.100498 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 10:33:55.100498 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:33:55.100498 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 284/569] 10.222.0.15 () {56 vars in 1102 bytes} [Sun Sep 11 10:33:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:55.100498 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497d6b0c-0d17-4bd8-aa10-a5c5ca2c0c88 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:55.110642 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab445dd6-6991-43ff-80e7-25362bc571f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:55.111248 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab445dd6-6991-43ff-80e7-25362bc571f8 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:55.111556 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab445dd6-6991-43ff-80e7-25362bc571f8 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:55.111831 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab445dd6-6991-43ff-80e7-25362bc571f8 None admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:55.112487 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab445dd6-6991-43ff-80e7-25362bc571f8 None admin] RBAC: Adding query filter params (name=service) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:55.112788 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab445dd6-6991-43ff-80e7-25362bc571f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:55.114144 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab445dd6-6991-43ff-80e7-25362bc571f8 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:55.122609 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 286/570] 10.222.0.15 () {56 vars in 1103 bytes} [Sun Sep 11 10:33:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:55.123376 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab445dd6-6991-43ff-80e7-25362bc571f8 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:55.135618 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:55.203203 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497d6b0c-0d17-4bd8-aa10-a5c5ca2c0c88 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:55.222759 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab445dd6-6991-43ff-80e7-25362bc571f8 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:55.224282 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:55.237241 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3558bd-cb14-4cf9-9388-92f14d8e44e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:55.237865 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3558bd-cb14-4cf9-9388-92f14d8e44e4 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:55.238133 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3558bd-cb14-4cf9-9388-92f14d8e44e4 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:55.238404 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3558bd-cb14-4cf9-9388-92f14d8e44e4 None admin] PATH_INFO: `/v3/users/glance` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:55.245638 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3558bd-cb14-4cf9-9388-92f14d8e44e4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:55.247061 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3558bd-cb14-4cf9-9388-92f14d8e44e4 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:55.258014 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b038a1ac-6c1c-4966-82d5-4cae25cce6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:55.258475 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-cd3558bd-cb14-4cf9-9388-92f14d8e44e4 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 11 10:33:55.258475 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:55.258475 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:55.258475 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:55.258475 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:55.258475 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:55.258475 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:55.258475 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:55.258475 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:55.258475 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:55.258475 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:55.258475 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:55.258475 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 10:33:55.258475 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 10:33:55.258475 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 10:33:55.258475 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 10:33:55.259617 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:55.259617 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:55.259617 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 10:33:55.259617 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:33:55.259617 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 10:33:55.259617 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:33:55.259617 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:55.259617 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:55.259617 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:55.259617 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:55.259617 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:55.259617 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:55.259617 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:55.259617 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:55.259617 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:55.259617 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:55.259617 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:55.259617 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:55.260930 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:55.260930 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:55.260930 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 10:33:55.260930 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 10:33:55.260930 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 10:33:55.260930 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 10:33:55.260930 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 10:33:55.260930 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 10:33:55.260930 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 11 10:33:55.260930 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:33:55.261486 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b038a1ac-6c1c-4966-82d5-4cae25cce6cf None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:55.261486 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b038a1ac-6c1c-4966-82d5-4cae25cce6cf None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:55.261486 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b038a1ac-6c1c-4966-82d5-4cae25cce6cf None admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:55.261757 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 285/571] 10.222.0.15 () {56 vars in 1099 bytes} [Sun Sep 11 10:33:55 2022] GET /identity/v3/users/glance => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:55.261757 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3558bd-cb14-4cf9-9388-92f14d8e44e4 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:55.263737 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b038a1ac-6c1c-4966-82d5-4cae25cce6cf None admin] RBAC: Adding query filter params (name=service) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:55.264943 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b038a1ac-6c1c-4966-82d5-4cae25cce6cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:55.266373 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b038a1ac-6c1c-4966-82d5-4cae25cce6cf None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:55.275024 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 287/572] 10.222.0.15 () {56 vars in 1103 bytes} [Sun Sep 11 10:33:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:55.275836 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b038a1ac-6c1c-4966-82d5-4cae25cce6cf None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:55.360364 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:55.366611 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3558bd-cb14-4cf9-9388-92f14d8e44e4 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:55.378295 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b038a1ac-6c1c-4966-82d5-4cae25cce6cf None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:55.399573 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acec4f6f-41d5-4293-8306-2e1ed91c10dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:55.400174 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec4f6f-41d5-4293-8306-2e1ed91c10dc None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:55.400453 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec4f6f-41d5-4293-8306-2e1ed91c10dc None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:55.400725 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec4f6f-41d5-4293-8306-2e1ed91c10dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:55.401434 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acec4f6f-41d5-4293-8306-2e1ed91c10dc None admin] RBAC: Adding query filter params (role.id=3cb7ebbc707d453586b3d0f4ab3cf103, scope.project.id=5e1f845f99f04202b83621cb65e0adc2, user.id=cc7ca566e66a45b29e3d02f47bf84981) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:55.401739 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acec4f6f-41d5-4293-8306-2e1ed91c10dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:55.403166 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acec4f6f-41d5-4293-8306-2e1ed91c10dc None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:55.408341 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 286/573] 10.222.0.15 () {56 vars in 1374 bytes} [Sun Sep 11 10:33:55 2022] GET /identity/v3/role_assignments?user.id=cc7ca566e66a45b29e3d02f47bf84981&scope.project.id=5e1f845f99f04202b83621cb65e0adc2&role.id=3cb7ebbc707d453586b3d0f4ab3cf103 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:55.409108 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acec4f6f-41d5-4293-8306-2e1ed91c10dc None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:55.425362 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dcb38f-153a-4bd4-a8bd-0634bf80f4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:55.425999 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dcb38f-153a-4bd4-a8bd-0634bf80f4c8 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:55.426270 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dcb38f-153a-4bd4-a8bd-0634bf80f4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:55.426545 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dcb38f-153a-4bd4-a8bd-0634bf80f4c8 None admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:55.427241 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dcb38f-153a-4bd4-a8bd-0634bf80f4c8 None admin] RBAC: Adding query filter params (name=service) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:55.427589 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dcb38f-153a-4bd4-a8bd-0634bf80f4c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:55.429174 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dcb38f-153a-4bd4-a8bd-0634bf80f4c8 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:55.437965 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 288/574] 10.222.0.15 () {56 vars in 1103 bytes} [Sun Sep 11 10:33:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:55.438826 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dcb38f-153a-4bd4-a8bd-0634bf80f4c8 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:55.470686 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:55.527689 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acec4f6f-41d5-4293-8306-2e1ed91c10dc None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:55.557013 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dcb38f-153a-4bd4-a8bd-0634bf80f4c8 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:55.565057 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a255502a-baa8-4ba9-ae1c-b3a66be380db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:55.565625 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a255502a-baa8-4ba9-ae1c-b3a66be380db None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:55.565901 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a255502a-baa8-4ba9-ae1c-b3a66be380db None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:55.566170 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a255502a-baa8-4ba9-ae1c-b3a66be380db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:55.578535 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a255502a-baa8-4ba9-ae1c-b3a66be380db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:55.580058 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a255502a-baa8-4ba9-ae1c-b3a66be380db None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:55.596527 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-a255502a-baa8-4ba9-ae1c-b3a66be380db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:33:55.596527 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:55.596527 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:55.596527 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:55.596527 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:55.596527 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:55.596527 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:55.596527 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:55.596527 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:55.596527 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:55.596527 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:55.596527 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:55.596527 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:33:55.596527 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:33:55.596527 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:33:55.596527 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:33:55.597530 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:55.597530 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:55.597530 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:55.597530 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:55.597530 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:55.597530 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:55.597530 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:55.597530 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:55.597530 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:55.597530 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:55.597530 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:55.597530 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:55.597530 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:55.597530 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:55.597530 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:55.597530 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:55.597530 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:33:55.599090 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:33:55.599090 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:33:55.599090 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:33:55.599090 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 287/575] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:33:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:55.599090 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a255502a-baa8-4ba9-ae1c-b3a66be380db None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:55.606444 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eafa4e-4be1-47e6-9ddf-947cfb7cb4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:55.607069 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eafa4e-4be1-47e6-9ddf-947cfb7cb4e2 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:55.607343 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eafa4e-4be1-47e6-9ddf-947cfb7cb4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:55.607638 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eafa4e-4be1-47e6-9ddf-947cfb7cb4e2 None admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:55.608371 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eafa4e-4be1-47e6-9ddf-947cfb7cb4e2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:55.608671 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eafa4e-4be1-47e6-9ddf-947cfb7cb4e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:55.610087 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eafa4e-4be1-47e6-9ddf-947cfb7cb4e2 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:55.616599 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:55.635630 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 289/576] 10.222.0.15 () {56 vars in 1137 bytes} [Sun Sep 11 10:33:55 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:55.635630 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eafa4e-4be1-47e6-9ddf-947cfb7cb4e2 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:55.709135 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a255502a-baa8-4ba9-ae1c-b3a66be380db None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:55.732905 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:55.745807 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f6023f-4998-421f-bbcf-9e21fd74e5d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:55.746443 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f6023f-4998-421f-bbcf-9e21fd74e5d7 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:55.746720 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f6023f-4998-421f-bbcf-9e21fd74e5d7 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:55.746989 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f6023f-4998-421f-bbcf-9e21fd74e5d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:55.751841 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eafa4e-4be1-47e6-9ddf-947cfb7cb4e2 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:55.757889 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f6023f-4998-421f-bbcf-9e21fd74e5d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:55.759337 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f6023f-4998-421f-bbcf-9e21fd74e5d7 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:55.774268 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-72f6023f-4998-421f-bbcf-9e21fd74e5d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:33:55.774268 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:55.774268 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:55.774268 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:55.774268 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:55.774268 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:55.774268 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:55.774268 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:55.774268 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:55.774268 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:55.774268 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:55.774268 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:55.774268 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:33:55.774268 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:33:55.774268 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:33:55.774268 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:33:55.775235 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:55.775235 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:55.775235 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:55.775235 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:55.775235 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:55.775235 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:55.775235 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:55.775235 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:55.775235 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:55.775235 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:55.775235 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:55.775235 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:55.775235 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:55.775235 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:55.775235 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:55.775235 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:55.775235 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:33:55.776172 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:33:55.776172 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:33:55.776172 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:33:55.776172 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 288/577] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:33:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:55.776172 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f6023f-4998-421f-bbcf-9e21fd74e5d7 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:55.794210 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cb67e2-3170-4634-846c-8c3aa0ab226a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:55.794869 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb67e2-3170-4634-846c-8c3aa0ab226a None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:55.795147 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb67e2-3170-4634-846c-8c3aa0ab226a None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:55.795414 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb67e2-3170-4634-846c-8c3aa0ab226a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:55.806224 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cb67e2-3170-4634-846c-8c3aa0ab226a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:55.807682 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cb67e2-3170-4634-846c-8c3aa0ab226a None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:55.825026 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-41cb67e2-3170-4634-846c-8c3aa0ab226a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:33:55.825026 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:55.825026 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:55.825026 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:55.825026 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:55.825026 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:55.825026 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:55.825026 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:55.825026 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:55.825026 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:55.825026 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:55.825026 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:55.825026 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:33:55.825026 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:33:55.825026 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:33:55.825026 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:33:55.825980 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:55.825980 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:55.825980 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:55.825980 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:55.825980 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:55.825980 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:55.825980 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:55.825980 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:55.825980 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:55.825980 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:55.825980 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:55.825980 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:55.825980 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:55.825980 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:55.825980 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:55.825980 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:55.825980 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:33:55.827676 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:33:55.827676 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:33:55.827676 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:33:55.827676 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 290/578] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:33:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:55.827676 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cb67e2-3170-4634-846c-8c3aa0ab226a None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:55.848512 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:55.894595 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f6023f-4998-421f-bbcf-9e21fd74e5d7 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:55.930980 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055299d8-b3e3-46a0-934a-d01685a24d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:55.931613 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055299d8-b3e3-46a0-934a-d01685a24d76 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:55.931892 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055299d8-b3e3-46a0-934a-d01685a24d76 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:55.932160 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055299d8-b3e3-46a0-934a-d01685a24d76 None admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:55.932787 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055299d8-b3e3-46a0-934a-d01685a24d76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:55.933086 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055299d8-b3e3-46a0-934a-d01685a24d76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:55.934438 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055299d8-b3e3-46a0-934a-d01685a24d76 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:55.939805 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cb67e2-3170-4634-846c-8c3aa0ab226a None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:55.947767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 289/579] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:33:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:55.948399 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055299d8-b3e3-46a0-934a-d01685a24d76 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:55.982159 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a14e32d-9d73-4982-b1ad-ca7a074bec21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:55.982788 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a14e32d-9d73-4982-b1ad-ca7a074bec21 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:55.983064 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a14e32d-9d73-4982-b1ad-ca7a074bec21 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:55.983332 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a14e32d-9d73-4982-b1ad-ca7a074bec21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:55.986368 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:55.996294 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a14e32d-9d73-4982-b1ad-ca7a074bec21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:55.997859 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a14e32d-9d73-4982-b1ad-ca7a074bec21 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:56.014147 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-2a14e32d-9d73-4982-b1ad-ca7a074bec21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:33:56.014147 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:56.014147 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:56.014147 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:56.014147 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:56.014147 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:56.014147 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:56.014147 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:56.014147 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:56.014147 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:56.014147 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:56.014147 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:56.014147 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:33:56.014147 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:33:56.014147 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:33:56.014147 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:33:56.015298 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:56.015298 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:56.015298 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:56.015298 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:56.015298 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:56.015298 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:56.015298 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:56.015298 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:56.015298 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:56.015298 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:56.015298 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:56.015298 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:56.015298 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:56.015298 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:56.015298 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:56.015298 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:56.015298 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:33:56.016329 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:33:56.016329 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:33:56.016329 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:33:56.016329 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 291/580] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:33:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:56.016970 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a14e32d-9d73-4982-b1ad-ca7a074bec21 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:56.055791 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055299d8-b3e3-46a0-934a-d01685a24d76 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:56.086392 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:56.100157 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af20cb5-7b31-4133-8b4e-b5511c8edc06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:56.101826 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af20cb5-7b31-4133-8b4e-b5511c8edc06 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:56.102517 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af20cb5-7b31-4133-8b4e-b5511c8edc06 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:56.102801 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af20cb5-7b31-4133-8b4e-b5511c8edc06 None admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:56.103472 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af20cb5-7b31-4133-8b4e-b5511c8edc06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:56.103780 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af20cb5-7b31-4133-8b4e-b5511c8edc06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:56.105174 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af20cb5-7b31-4133-8b4e-b5511c8edc06 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:56.119469 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 290/581] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:33:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:56.119469 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af20cb5-7b31-4133-8b4e-b5511c8edc06 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:56.127854 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a14e32d-9d73-4982-b1ad-ca7a074bec21 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:56.165505 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e9b5ae-b3ae-4fd5-b530-67febf61a993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:56.166120 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e9b5ae-b3ae-4fd5-b530-67febf61a993 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:56.166403 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e9b5ae-b3ae-4fd5-b530-67febf61a993 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:56.166678 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e9b5ae-b3ae-4fd5-b530-67febf61a993 None admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:56.167324 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e9b5ae-b3ae-4fd5-b530-67febf61a993 None admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:56.167666 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e9b5ae-b3ae-4fd5-b530-67febf61a993 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:56.169071 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e9b5ae-b3ae-4fd5-b530-67febf61a993 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:56.186099 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 292/582] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:33:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:56.186099 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e9b5ae-b3ae-4fd5-b530-67febf61a993 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:56.227633 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af20cb5-7b31-4133-8b4e-b5511c8edc06 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:56.237538 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:56.262562 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c7948a-7052-4bd0-919b-4ac4f4a360ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:56.263189 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c7948a-7052-4bd0-919b-4ac4f4a360ff None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:56.263502 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c7948a-7052-4bd0-919b-4ac4f4a360ff None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:56.263778 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c7948a-7052-4bd0-919b-4ac4f4a360ff None admin] PATH_INFO: `/v3/users/nova` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:56.271286 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c7948a-7052-4bd0-919b-4ac4f4a360ff None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:56.272928 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c7948a-7052-4bd0-919b-4ac4f4a360ff None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:56.284973 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-f8c7948a-7052-4bd0-919b-4ac4f4a360ff None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 11 10:33:56.284973 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:56.284973 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:56.284973 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:56.284973 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:56.284973 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:56.284973 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:56.284973 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:56.284973 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:56.284973 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:56.284973 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:56.284973 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:56.284973 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 10:33:56.284973 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 10:33:56.284973 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 10:33:56.284973 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 10:33:56.286022 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:56.286022 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:56.286022 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 10:33:56.286022 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:33:56.286022 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 10:33:56.286022 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:33:56.286022 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:56.286022 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:56.286022 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:56.286022 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:56.286022 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:56.286022 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:56.286022 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:56.286022 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:56.286022 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:56.286022 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:56.286022 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:56.286022 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:56.287029 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:56.287029 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:56.287029 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 10:33:56.287029 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 10:33:56.287029 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 10:33:56.287029 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 10:33:56.287029 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 10:33:56.287029 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 10:33:56.287029 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 11 10:33:56.287029 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:33:56.287029 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 291/583] 10.222.0.15 () {56 vars in 1093 bytes} [Sun Sep 11 10:33:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:56.287029 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c7948a-7052-4bd0-919b-4ac4f4a360ff None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:56.304269 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e9b5ae-b3ae-4fd5-b530-67febf61a993 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:56.340759 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:56.347181 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d8960e-8715-4e33-bc89-a945039ce77f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:56.347821 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d8960e-8715-4e33-bc89-a945039ce77f None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:56.348099 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d8960e-8715-4e33-bc89-a945039ce77f None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:56.348371 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d8960e-8715-4e33-bc89-a945039ce77f None admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:56.349004 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d8960e-8715-4e33-bc89-a945039ce77f None admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:56.349307 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d8960e-8715-4e33-bc89-a945039ce77f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:56.350682 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d8960e-8715-4e33-bc89-a945039ce77f None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:56.367846 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 293/584] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:33:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:56.367846 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d8960e-8715-4e33-bc89-a945039ce77f None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:56.410025 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c7948a-7052-4bd0-919b-4ac4f4a360ff None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:56.445524 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12cd77e-d63f-4a40-9a13-837cf705e5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:56.446301 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12cd77e-d63f-4a40-9a13-837cf705e5d6 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:56.446605 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12cd77e-d63f-4a40-9a13-837cf705e5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:56.446877 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12cd77e-d63f-4a40-9a13-837cf705e5d6 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:56.454257 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12cd77e-d63f-4a40-9a13-837cf705e5d6 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:56.455723 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12cd77e-d63f-4a40-9a13-837cf705e5d6 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:56.467773 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-d12cd77e-d63f-4a40-9a13-837cf705e5d6 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 11 10:33:56.467773 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:56.467773 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:56.467773 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:56.467773 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:56.467773 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:56.467773 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:56.467773 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:56.467773 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:56.467773 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:56.467773 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:56.467773 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:56.467773 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 10:33:56.467773 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 10:33:56.467773 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 10:33:56.467773 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 10:33:56.468766 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:56.468766 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:56.468766 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 10:33:56.468766 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:33:56.468766 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 10:33:56.468766 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:33:56.468766 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:56.468766 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:56.468766 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:56.468766 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:56.468766 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:56.468766 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:56.468766 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:56.468766 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:56.468766 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:56.468766 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:56.468766 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:56.468766 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:56.469713 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:56.469713 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:56.469713 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 10:33:56.469713 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 10:33:56.469713 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 10:33:56.469713 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 10:33:56.469713 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 10:33:56.469713 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 10:33:56.469713 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 11 10:33:56.469713 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:33:56.473568 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 292/585] 10.222.0.15 () {56 vars in 1102 bytes} [Sun Sep 11 10:33:56 2022] GET /identity/v3/users/neutron => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:56.474380 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12cd77e-d63f-4a40-9a13-837cf705e5d6 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:56.481966 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d8960e-8715-4e33-bc89-a945039ce77f None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:56.493422 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:56.521299 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5f75b6-592c-4d05-b4f0-8f4c55f03e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:56.522120 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f75b6-592c-4d05-b4f0-8f4c55f03e6c None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:56.522240 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f75b6-592c-4d05-b4f0-8f4c55f03e6c None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:56.522477 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f75b6-592c-4d05-b4f0-8f4c55f03e6c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:56.530483 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5f75b6-592c-4d05-b4f0-8f4c55f03e6c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:56.531961 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5f75b6-592c-4d05-b4f0-8f4c55f03e6c None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:56.543104 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-db5f75b6-592c-4d05-b4f0-8f4c55f03e6c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 11 10:33:56.543104 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:56.543104 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:56.543104 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:56.543104 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:56.543104 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:56.543104 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:56.543104 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:56.543104 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:56.543104 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:56.543104 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:56.543104 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:56.543104 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 10:33:56.543104 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 10:33:56.543104 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 10:33:56.543104 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 10:33:56.544095 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:56.544095 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:56.544095 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 10:33:56.544095 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:33:56.544095 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 10:33:56.544095 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:33:56.544095 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:56.544095 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:56.544095 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:56.544095 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:56.544095 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:56.544095 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:56.544095 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:56.544095 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:56.544095 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:56.544095 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:56.544095 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:56.544095 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:56.545042 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:56.545042 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:56.545042 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 10:33:56.545042 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 10:33:56.545042 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 10:33:56.545042 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 10:33:56.545042 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 10:33:56.545042 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 10:33:56.545042 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 11 10:33:56.545042 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:33:56.545042 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 294/586] 10.222.0.15 () {56 vars in 1099 bytes} [Sun Sep 11 10:33:56 2022] GET /identity/v3/users/cinder => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:56.545042 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5f75b6-592c-4d05-b4f0-8f4c55f03e6c None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:56.584299 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12cd77e-d63f-4a40-9a13-837cf705e5d6 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:56.587033 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:56.621791 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94246183-4c4e-409e-9df7-c763a3322fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:56.622315 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94246183-4c4e-409e-9df7-c763a3322fb0 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:56.622588 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94246183-4c4e-409e-9df7-c763a3322fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:56.622857 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94246183-4c4e-409e-9df7-c763a3322fb0 None admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:56.623596 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94246183-4c4e-409e-9df7-c763a3322fb0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:56.623901 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94246183-4c4e-409e-9df7-c763a3322fb0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:56.625313 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94246183-4c4e-409e-9df7-c763a3322fb0 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:56.648000 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 293/587] 10.222.0.15 () {56 vars in 1133 bytes} [Sun Sep 11 10:33:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:56.648744 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94246183-4c4e-409e-9df7-c763a3322fb0 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:56.655287 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5f75b6-592c-4d05-b4f0-8f4c55f03e6c None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:56.693514 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c81236a-17f9-4c40-9bab-ba70c3a69603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:56.694245 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c81236a-17f9-4c40-9bab-ba70c3a69603 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:56.695229 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c81236a-17f9-4c40-9bab-ba70c3a69603 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:56.695595 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c81236a-17f9-4c40-9bab-ba70c3a69603 None admin] PATH_INFO: `/v3/projects/service` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:56.702468 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c81236a-17f9-4c40-9bab-ba70c3a69603 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:56.704023 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c81236a-17f9-4c40-9bab-ba70c3a69603 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:56.715507 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-9c81236a-17f9-4c40-9bab-ba70c3a69603 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:33:56.715507 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:56.715507 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:56.715507 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:56.715507 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:56.715507 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:56.715507 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:56.715507 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:56.715507 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:56.715507 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:56.715507 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:56.715507 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:56.715507 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 10:33:56.715507 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 10:33:56.715507 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 10:33:56.716597 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 10:33:56.716597 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:56.716597 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:56.716597 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:56.716597 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:56.716597 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:56.716597 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:56.716597 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:56.716597 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:56.716597 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:56.716597 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:56.716597 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:56.716597 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:56.716597 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:56.716597 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:56.716597 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:56.716597 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:56.717780 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 10:33:56.717780 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 10:33:56.717780 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 10:33:56.717780 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 10:33:56.717780 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 10:33:56.717780 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 10:33:56.717780 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:33:56.717780 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:33:56.718267 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 295/588] 10.222.0.15 () {56 vars in 1111 bytes} [Sun Sep 11 10:33:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:56.718814 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c81236a-17f9-4c40-9bab-ba70c3a69603 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:56.755322 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:56.761243 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94246183-4c4e-409e-9df7-c763a3322fb0 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:56.796137 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa653f5d-2d16-447e-ab72-5f0757f003fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:56.796777 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa653f5d-2d16-447e-ab72-5f0757f003fc None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:56.797063 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa653f5d-2d16-447e-ab72-5f0757f003fc None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:56.797334 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa653f5d-2d16-447e-ab72-5f0757f003fc None admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:56.798047 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa653f5d-2d16-447e-ab72-5f0757f003fc None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:56.798356 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa653f5d-2d16-447e-ab72-5f0757f003fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:56.799816 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa653f5d-2d16-447e-ab72-5f0757f003fc None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:56.821909 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 294/589] 10.222.0.15 () {56 vars in 1139 bytes} [Sun Sep 11 10:33:56 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:56.821909 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa653f5d-2d16-447e-ab72-5f0757f003fc None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:56.840141 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c81236a-17f9-4c40-9bab-ba70c3a69603 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:56.843671 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:56.878759 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc54ce9-6999-4006-8122-2adb7f1b2156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:56.879286 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc54ce9-6999-4006-8122-2adb7f1b2156 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:56.879604 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc54ce9-6999-4006-8122-2adb7f1b2156 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:56.879869 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc54ce9-6999-4006-8122-2adb7f1b2156 None admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:56.880640 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc54ce9-6999-4006-8122-2adb7f1b2156 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:56.880947 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc54ce9-6999-4006-8122-2adb7f1b2156 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:56.882528 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc54ce9-6999-4006-8122-2adb7f1b2156 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:56.904749 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 296/590] 10.222.0.15 () {56 vars in 1137 bytes} [Sun Sep 11 10:33:56 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:56.904749 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc54ce9-6999-4006-8122-2adb7f1b2156 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:56.932120 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa653f5d-2d16-447e-ab72-5f0757f003fc None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:56.966813 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a659358b-5924-4b91-9831-168d12916765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:56.967494 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a659358b-5924-4b91-9831-168d12916765 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:56.967744 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a659358b-5924-4b91-9831-168d12916765 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:56.968016 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a659358b-5924-4b91-9831-168d12916765 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:56.978729 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a659358b-5924-4b91-9831-168d12916765 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:56.980199 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a659358b-5924-4b91-9831-168d12916765 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:56.994458 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:56.994621 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-a659358b-5924-4b91-9831-168d12916765 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:33:56.994621 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:56.994621 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:56.994621 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:56.994621 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:56.994621 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:56.994621 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:56.994621 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:56.994621 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:56.994621 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:56.994621 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:56.994621 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:56.994621 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:33:56.994621 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:33:56.994621 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:33:56.994621 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:33:57.000824 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:57.000824 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:57.000824 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:57.000824 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:57.000824 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:57.000824 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:57.000824 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:57.000824 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:57.000824 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:57.000824 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:57.000824 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:57.000824 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:57.000824 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:57.000824 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:57.000824 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:57.000824 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:57.000824 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:33:57.002007 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:33:57.002007 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:33:57.002007 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:33:57.002007 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 295/591] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:33:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:57.002007 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a659358b-5924-4b91-9831-168d12916765 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:57.019135 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc54ce9-6999-4006-8122-2adb7f1b2156 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:57.057794 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37821eb2-dbeb-43ff-b86b-e4dffa0ec0ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:57.058500 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37821eb2-dbeb-43ff-b86b-e4dffa0ec0ef None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:57.058854 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37821eb2-dbeb-43ff-b86b-e4dffa0ec0ef None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:57.059215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37821eb2-dbeb-43ff-b86b-e4dffa0ec0ef None admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:57.060002 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37821eb2-dbeb-43ff-b86b-e4dffa0ec0ef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:57.060428 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37821eb2-dbeb-43ff-b86b-e4dffa0ec0ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:57.061768 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37821eb2-dbeb-43ff-b86b-e4dffa0ec0ef None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:57.074635 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 297/592] 10.222.0.15 () {56 vars in 1148 bytes} [Sun Sep 11 10:33:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:57.076702 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37821eb2-dbeb-43ff-b86b-e4dffa0ec0ef None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:57.082245 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:57.117761 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a659358b-5924-4b91-9831-168d12916765 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:57.155280 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea072264-9771-4dd5-bda7-2e6abf3e4019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:57.156201 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea072264-9771-4dd5-bda7-2e6abf3e4019 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:57.156201 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea072264-9771-4dd5-bda7-2e6abf3e4019 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:57.156396 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea072264-9771-4dd5-bda7-2e6abf3e4019 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:57.170066 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea072264-9771-4dd5-bda7-2e6abf3e4019 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:57.171543 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea072264-9771-4dd5-bda7-2e6abf3e4019 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:57.187769 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-ea072264-9771-4dd5-bda7-2e6abf3e4019 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:33:57.187769 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:57.187769 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:57.187769 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:57.187769 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:57.187769 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:57.187769 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:57.187769 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:57.187769 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:57.187769 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:57.187769 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:57.187769 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:57.187769 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:33:57.187769 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:33:57.187769 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:33:57.187769 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:33:57.188767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:57.188767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:57.188767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:57.188767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:57.188767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:57.188767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:57.188767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:57.188767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:57.188767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:57.188767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:57.188767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:57.188767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:57.188767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:57.188767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:57.188767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:57.188767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:57.188767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:33:57.189659 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:33:57.189659 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:33:57.189659 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:33:57.189659 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 296/593] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:33:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:57.189659 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea072264-9771-4dd5-bda7-2e6abf3e4019 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:57.203298 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37821eb2-dbeb-43ff-b86b-e4dffa0ec0ef None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:57.246884 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479336f5-8b64-4091-85a1-31c524ee35a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:57.247844 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479336f5-8b64-4091-85a1-31c524ee35a7 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:57.248123 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479336f5-8b64-4091-85a1-31c524ee35a7 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:57.248392 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479336f5-8b64-4091-85a1-31c524ee35a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:57.263278 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479336f5-8b64-4091-85a1-31c524ee35a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:57.265251 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479336f5-8b64-4091-85a1-31c524ee35a7 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:57.272506 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:57.286626 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-479336f5-8b64-4091-85a1-31c524ee35a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:33:57.286626 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:57.286626 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:57.286626 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:57.286626 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:57.286626 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:57.286626 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:57.286626 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:57.286626 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:57.286626 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:57.286626 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:57.286626 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:57.286626 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:33:57.286626 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:33:57.286626 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:33:57.286626 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:33:57.295036 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:57.295036 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:57.295036 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:57.295036 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:57.295036 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:57.295036 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:57.295036 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:57.295036 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:57.295036 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:57.295036 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:57.295036 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:57.295036 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:57.295036 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:57.295036 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:57.295036 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:57.295036 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:57.295036 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:33:57.296204 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:33:57.296204 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:33:57.296204 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:33:57.296204 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 298/594] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:33:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:57.296204 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479336f5-8b64-4091-85a1-31c524ee35a7 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:57.324209 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea072264-9771-4dd5-bda7-2e6abf3e4019 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:57.355401 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:57.369161 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17e9cf1-596c-4e9d-a08a-053550a43612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:57.369779 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17e9cf1-596c-4e9d-a08a-053550a43612 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:57.370058 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17e9cf1-596c-4e9d-a08a-053550a43612 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:57.370329 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17e9cf1-596c-4e9d-a08a-053550a43612 None admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:57.371003 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17e9cf1-596c-4e9d-a08a-053550a43612 None admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:57.371305 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17e9cf1-596c-4e9d-a08a-053550a43612 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:57.372713 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17e9cf1-596c-4e9d-a08a-053550a43612 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:57.394221 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 297/595] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:33:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:57.394221 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17e9cf1-596c-4e9d-a08a-053550a43612 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:57.427705 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479336f5-8b64-4091-85a1-31c524ee35a7 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:57.469174 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc75fd40-f2fc-4efe-9ee7-74d02ff02d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:57.469847 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc75fd40-f2fc-4efe-9ee7-74d02ff02d79 None admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:57.470148 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc75fd40-f2fc-4efe-9ee7-74d02ff02d79 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:57.470382 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc75fd40-f2fc-4efe-9ee7-74d02ff02d79 None admin] PATH_INFO: `/v3/projects/5e1f845f99f04202b83621cb65e0adc2/users/1632da5fbe384b5c849be1d4d86c0f1d/roles/3cb7ebbc707d453586b3d0f4ab3cf103` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:57.492932 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc75fd40-f2fc-4efe-9ee7-74d02ff02d79 None admin] RBAC: Authorizing `identity:create_grant(project_id=5e1f845f99f04202b83621cb65e0adc2, user_id=1632da5fbe384b5c849be1d4d86c0f1d, role_id=3cb7ebbc707d453586b3d0f4ab3cf103)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:57.495107 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc75fd40-f2fc-4efe-9ee7-74d02ff02d79 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:57.519456 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 299/596] 10.222.0.15 () {58 vars in 1439 bytes} [Sun Sep 11 10:33:57 2022] PUT /identity/v3/projects/5e1f845f99f04202b83621cb65e0adc2/users/1632da5fbe384b5c849be1d4d86c0f1d/roles/3cb7ebbc707d453586b3d0f4ab3cf103 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:33:57.519456 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc75fd40-f2fc-4efe-9ee7-74d02ff02d79 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:57.533407 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17e9cf1-596c-4e9d-a08a-053550a43612 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:57.586309 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:57.597601 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05b097f-6e57-4f4a-b2c5-a8e424e0f425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:57.598239 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05b097f-6e57-4f4a-b2c5-a8e424e0f425 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:57.598702 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05b097f-6e57-4f4a-b2c5-a8e424e0f425 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:57.599048 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05b097f-6e57-4f4a-b2c5-a8e424e0f425 None admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:57.601277 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05b097f-6e57-4f4a-b2c5-a8e424e0f425 None admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:57.602213 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05b097f-6e57-4f4a-b2c5-a8e424e0f425 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:57.604132 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05b097f-6e57-4f4a-b2c5-a8e424e0f425 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:57.630273 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 298/597] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:33:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:57.631358 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05b097f-6e57-4f4a-b2c5-a8e424e0f425 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:57.666233 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc75fd40-f2fc-4efe-9ee7-74d02ff02d79 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:57.697781 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:57.728209 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8223b8ae-2b37-4846-9437-970081182a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:57.728727 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223b8ae-2b37-4846-9437-970081182a13 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:57.729902 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223b8ae-2b37-4846-9437-970081182a13 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:57.729902 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223b8ae-2b37-4846-9437-970081182a13 None admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:57.731093 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8223b8ae-2b37-4846-9437-970081182a13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:57.731093 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8223b8ae-2b37-4846-9437-970081182a13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:57.733605 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8223b8ae-2b37-4846-9437-970081182a13 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:57.753089 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 300/598] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:33:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:57.754574 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8223b8ae-2b37-4846-9437-970081182a13 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:57.793504 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05b097f-6e57-4f4a-b2c5-a8e424e0f425 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:57.836983 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d9c40b-e4c6-4415-b342-cabcbe433837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:57.837929 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d9c40b-e4c6-4415-b342-cabcbe433837 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:57.838117 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d9c40b-e4c6-4415-b342-cabcbe433837 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:57.838403 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d9c40b-e4c6-4415-b342-cabcbe433837 None admin] PATH_INFO: `/v3/projects/service` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:57.846101 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d9c40b-e4c6-4415-b342-cabcbe433837 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:57.847611 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d9c40b-e4c6-4415-b342-cabcbe433837 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:57.860143 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-e0d9c40b-e4c6-4415-b342-cabcbe433837 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:33:57.860143 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:57.860143 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:57.860143 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:57.860143 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:57.860143 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:57.860143 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:57.860143 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:57.860143 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:57.860143 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:57.860143 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:57.860143 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:57.860143 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 10:33:57.860143 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 10:33:57.860143 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 10:33:57.861175 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 10:33:57.861175 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:57.861175 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:57.861175 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:57.861175 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:57.861175 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:57.861175 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:57.861175 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:57.861175 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:57.861175 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:57.861175 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:57.861175 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:57.861175 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:57.861175 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:57.861175 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:57.861175 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:57.861175 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:57.862190 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 10:33:57.862190 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 10:33:57.862190 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 10:33:57.862190 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 10:33:57.862190 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 10:33:57.862190 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 10:33:57.862190 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:33:57.862190 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:33:57.862190 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 299/599] 10.222.0.15 () {56 vars in 1111 bytes} [Sun Sep 11 10:33:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:57.864707 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780b98a-d77f-4aa7-9e75-7ae09eb141b0 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:57.865173 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780b98a-d77f-4aa7-9e75-7ae09eb141b0 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:57.865918 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780b98a-d77f-4aa7-9e75-7ae09eb141b0 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:57.868464 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 300/600] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:33:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:33:57.869069 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a780b98a-d77f-4aa7-9e75-7ae09eb141b0 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:57.881173 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8223b8ae-2b37-4846-9437-970081182a13 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:57.895963 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:57.926535 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c530285-8118-42b7-9389-aecf5258b8f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:57.927180 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c530285-8118-42b7-9389-aecf5258b8f9 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:57.927484 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c530285-8118-42b7-9389-aecf5258b8f9 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:57.927757 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c530285-8118-42b7-9389-aecf5258b8f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:57.936214 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c530285-8118-42b7-9389-aecf5258b8f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:57.937669 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c530285-8118-42b7-9389-aecf5258b8f9 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:57.951752 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-9c530285-8118-42b7-9389-aecf5258b8f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:33:57.951752 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:57.951752 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:57.951752 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:57.951752 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:57.951752 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:57.951752 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:57.951752 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:57.951752 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:57.951752 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:57.951752 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:57.951752 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:57.951752 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 10:33:57.951752 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 10:33:57.951752 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 10:33:57.952700 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 10:33:57.952700 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:57.952700 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:57.952700 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:57.952700 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:57.952700 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:57.952700 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:57.952700 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:57.952700 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:57.952700 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:57.952700 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:57.952700 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:57.952700 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:57.952700 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:57.952700 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:57.952700 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:57.952700 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:57.953589 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 10:33:57.953589 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 10:33:57.953589 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 10:33:57.953589 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 10:33:57.953589 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 10:33:57.953589 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 10:33:57.953589 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:33:57.953589 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:33:57.959851 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 301/601] 10.222.0.15 () {56 vars in 1111 bytes} [Sun Sep 11 10:33:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:57.961325 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c530285-8118-42b7-9389-aecf5258b8f9 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:57.999726 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a780b98a-d77f-4aa7-9e75-7ae09eb141b0 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:58.002072 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:58.044475 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bfe742-9691-400f-bfd8-84037e90ab23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:58.045300 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bfe742-9691-400f-bfd8-84037e90ab23 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:58.045581 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bfe742-9691-400f-bfd8-84037e90ab23 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:58.045850 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bfe742-9691-400f-bfd8-84037e90ab23 None admin] PATH_INFO: `/v3/projects/service` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:58.053536 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bfe742-9691-400f-bfd8-84037e90ab23 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:58.055508 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bfe742-9691-400f-bfd8-84037e90ab23 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:58.067717 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-b5bfe742-9691-400f-bfd8-84037e90ab23 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:33:58.067717 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:58.067717 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:58.067717 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:58.067717 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:58.067717 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:58.067717 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:58.067717 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:58.067717 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:58.067717 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:58.067717 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:58.067717 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:58.067717 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 10:33:58.067717 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 10:33:58.067717 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 10:33:58.068688 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 10:33:58.068688 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:58.068688 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:58.068688 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:58.068688 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:58.068688 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:58.068688 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:58.068688 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:58.068688 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:58.068688 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:58.068688 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:58.068688 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:58.068688 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:58.068688 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:58.068688 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:58.068688 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:58.068688 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:58.071590 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 10:33:58.071590 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 10:33:58.071590 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 10:33:58.071590 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 10:33:58.071590 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 10:33:58.071590 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 10:33:58.071590 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:33:58.071590 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:33:58.073427 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 301/602] 10.222.0.15 () {56 vars in 1111 bytes} [Sun Sep 11 10:33:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:58.075582 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb594f-8243-4cf0-8151-93a801257ccc None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:58.075718 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb594f-8243-4cf0-8151-93a801257ccc None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:58.075976 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb594f-8243-4cf0-8151-93a801257ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:58.092364 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c530285-8118-42b7-9389-aecf5258b8f9 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:58.094852 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-83cb594f-8243-4cf0-8151-93a801257ccc None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:33:58.096078 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-83cb594f-8243-4cf0-8151-93a801257ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:33:58.134015 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46344139-64a6-4b19-bb6c-87763012b32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:58.134627 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46344139-64a6-4b19-bb6c-87763012b32a None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:58.134903 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46344139-64a6-4b19-bb6c-87763012b32a None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:58.135175 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46344139-64a6-4b19-bb6c-87763012b32a None admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:58.136561 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46344139-64a6-4b19-bb6c-87763012b32a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:58.136868 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46344139-64a6-4b19-bb6c-87763012b32a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:58.138303 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46344139-64a6-4b19-bb6c-87763012b32a None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:58.152928 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 302/603] 10.222.0.15 () {56 vars in 1148 bytes} [Sun Sep 11 10:33:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:58.154321 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46344139-64a6-4b19-bb6c-87763012b32a None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:58.172615 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 302/604] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:33:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:33:58.174188 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cb594f-8243-4cf0-8151-93a801257ccc None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:58.197413 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:58.293139 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46344139-64a6-4b19-bb6c-87763012b32a None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:58.331972 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:58.335627 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8245e3d4-79ce-497c-a3a4-537568de7583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:58.336496 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8245e3d4-79ce-497c-a3a4-537568de7583 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:58.336784 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8245e3d4-79ce-497c-a3a4-537568de7583 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:58.337059 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8245e3d4-79ce-497c-a3a4-537568de7583 None admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:58.337934 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8245e3d4-79ce-497c-a3a4-537568de7583 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:58.338341 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8245e3d4-79ce-497c-a3a4-537568de7583 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:58.339825 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8245e3d4-79ce-497c-a3a4-537568de7583 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:58.350259 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cb594f-8243-4cf0-8151-93a801257ccc None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:58.353068 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 303/605] 10.222.0.15 () {56 vars in 1148 bytes} [Sun Sep 11 10:33:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:58.358773 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8245e3d4-79ce-497c-a3a4-537568de7583 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:58.406571 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be991175-2b84-493a-89c3-8c8c20608cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:58.407759 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be991175-2b84-493a-89c3-8c8c20608cc9 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:58.408200 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be991175-2b84-493a-89c3-8c8c20608cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:58.408858 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be991175-2b84-493a-89c3-8c8c20608cc9 None admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:58.410230 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be991175-2b84-493a-89c3-8c8c20608cc9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:58.410679 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be991175-2b84-493a-89c3-8c8c20608cc9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:58.412809 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be991175-2b84-493a-89c3-8c8c20608cc9 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:58.436041 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 303/606] 10.222.0.15 () {56 vars in 1148 bytes} [Sun Sep 11 10:33:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:58.436416 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be991175-2b84-493a-89c3-8c8c20608cc9 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:58.461143 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:58.479681 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8245e3d4-79ce-497c-a3a4-537568de7583 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:58.518962 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c01447-a60c-4b95-91d4-4896f3e9cb55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:58.520171 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c01447-a60c-4b95-91d4-4896f3e9cb55 None admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:58.520171 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c01447-a60c-4b95-91d4-4896f3e9cb55 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:58.520171 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c01447-a60c-4b95-91d4-4896f3e9cb55 None admin] PATH_INFO: `/v3/projects/5e1f845f99f04202b83621cb65e0adc2/users/2bd5356829b6454993bab9bad2f79fed/roles/3cb7ebbc707d453586b3d0f4ab3cf103` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:58.536807 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c01447-a60c-4b95-91d4-4896f3e9cb55 None admin] RBAC: Authorizing `identity:create_grant(project_id=5e1f845f99f04202b83621cb65e0adc2, user_id=2bd5356829b6454993bab9bad2f79fed, role_id=3cb7ebbc707d453586b3d0f4ab3cf103)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:58.538810 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c01447-a60c-4b95-91d4-4896f3e9cb55 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:58.553231 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be991175-2b84-493a-89c3-8c8c20608cc9 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:58.563665 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 304/607] 10.222.0.15 () {58 vars in 1439 bytes} [Sun Sep 11 10:33:58 2022] PUT /identity/v3/projects/5e1f845f99f04202b83621cb65e0adc2/users/2bd5356829b6454993bab9bad2f79fed/roles/3cb7ebbc707d453586b3d0f4ab3cf103 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:33:58.563665 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c01447-a60c-4b95-91d4-4896f3e9cb55 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:58.602825 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d58923f-c13c-4f76-8de1-26c034c16bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:58.603627 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d58923f-c13c-4f76-8de1-26c034c16bad None admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:58.603911 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d58923f-c13c-4f76-8de1-26c034c16bad None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:58.604183 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d58923f-c13c-4f76-8de1-26c034c16bad None admin] PATH_INFO: `/v3/projects/5e1f845f99f04202b83621cb65e0adc2/users/40072bc8dcb5454382a93e20b63f8594/roles/3cb7ebbc707d453586b3d0f4ab3cf103` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:58.621322 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d58923f-c13c-4f76-8de1-26c034c16bad None admin] RBAC: Authorizing `identity:create_grant(project_id=5e1f845f99f04202b83621cb65e0adc2, user_id=40072bc8dcb5454382a93e20b63f8594, role_id=3cb7ebbc707d453586b3d0f4ab3cf103)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:58.622982 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d58923f-c13c-4f76-8de1-26c034c16bad None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:58.641412 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 304/608] 10.222.0.15 () {58 vars in 1439 bytes} [Sun Sep 11 10:33:58 2022] PUT /identity/v3/projects/5e1f845f99f04202b83621cb65e0adc2/users/40072bc8dcb5454382a93e20b63f8594/roles/3cb7ebbc707d453586b3d0f4ab3cf103 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:33:58.642356 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d58923f-c13c-4f76-8de1-26c034c16bad None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:58.646952 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:58.698049 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c01447-a60c-4b95-91d4-4896f3e9cb55 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:58.787145 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bd5b07-be16-4ff6-b6ca-123572752eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:58.787862 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd5b07-be16-4ff6-b6ca-123572752eac None admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:58.788187 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd5b07-be16-4ff6-b6ca-123572752eac None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:58.788529 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd5b07-be16-4ff6-b6ca-123572752eac None admin] PATH_INFO: `/v3/projects/5e1f845f99f04202b83621cb65e0adc2/users/0bbb481abb8e43788e3412ac7d908ad3/roles/3cb7ebbc707d453586b3d0f4ab3cf103` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:58.810348 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:58.818624 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d58923f-c13c-4f76-8de1-26c034c16bad None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:58.824443 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bd5b07-be16-4ff6-b6ca-123572752eac None admin] RBAC: Authorizing `identity:create_grant(project_id=5e1f845f99f04202b83621cb65e0adc2, user_id=0bbb481abb8e43788e3412ac7d908ad3, role_id=3cb7ebbc707d453586b3d0f4ab3cf103)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:58.825929 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bd5b07-be16-4ff6-b6ca-123572752eac None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:58.843404 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 305/609] 10.222.0.15 () {58 vars in 1439 bytes} [Sun Sep 11 10:33:58 2022] PUT /identity/v3/projects/5e1f845f99f04202b83621cb65e0adc2/users/0bbb481abb8e43788e3412ac7d908ad3/roles/3cb7ebbc707d453586b3d0f4ab3cf103 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:33:58.864730 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de83e4d-b3c1-44d7-8ee3-2d3bd5a5555e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:58.865382 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de83e4d-b3c1-44d7-8ee3-2d3bd5a5555e None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:58.865662 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de83e4d-b3c1-44d7-8ee3-2d3bd5a5555e None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:58.865941 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de83e4d-b3c1-44d7-8ee3-2d3bd5a5555e None admin] PATH_INFO: `/v3/roles/service` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:58.881588 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de83e4d-b3c1-44d7-8ee3-2d3bd5a5555e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:33:58.887387 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-6de83e4d-b3c1-44d7-8ee3-2d3bd5a5555e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 10:33:58.887387 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:58.887387 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:58.887387 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:58.887387 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:58.887387 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:58.887387 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:58.887387 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:58.887387 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:58.887387 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:58.887387 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:58.887387 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:58.887387 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:33:58.887387 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:33:58.887387 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:33:58.887387 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:33:58.888386 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:33:58.888386 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:33:58.888386 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:33:58.888386 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:33:58.888386 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:58.888386 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:58.888386 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:58.888386 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:58.888386 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:58.888386 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:58.888386 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:58.888386 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:58.888386 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:58.888386 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:58.888386 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:58.888386 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:58.889256 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:58.889256 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:58.889256 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:58.889256 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:58.889256 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:33:58.889256 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:33:58.889256 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:33:58.889256 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:33:58.889256 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:33:58.889256 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:33:58.889256 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 10:33:58.889256 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:33:58.889256 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 305/610] 10.222.0.15 () {56 vars in 1102 bytes} [Sun Sep 11 10:33:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:58.894871 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bd5b07-be16-4ff6-b6ca-123572752eac None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:59.010694 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bd5b07-be16-4ff6-b6ca-123572752eac None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:59.049102 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eea46e8-4b35-4896-93e7-e40368d61306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:59.049784 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eea46e8-4b35-4896-93e7-e40368d61306 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:59.050072 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eea46e8-4b35-4896-93e7-e40368d61306 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:59.050343 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eea46e8-4b35-4896-93e7-e40368d61306 None admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:59.051024 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eea46e8-4b35-4896-93e7-e40368d61306 None admin] RBAC: Adding query filter params (name=service) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:59.051332 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eea46e8-4b35-4896-93e7-e40368d61306 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:59.052754 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eea46e8-4b35-4896-93e7-e40368d61306 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:59.067147 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 306/611] 10.222.0.15 () {56 vars in 1103 bytes} [Sun Sep 11 10:33:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:59.072516 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de83e4d-b3c1-44d7-8ee3-2d3bd5a5555e None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:59.153312 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:59.193797 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de83e4d-b3c1-44d7-8ee3-2d3bd5a5555e None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:59.231127 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d901f22d-0acf-467d-b0a6-11877c9c9893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:59.231796 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d901f22d-0acf-467d-b0a6-11877c9c9893 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:59.232075 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d901f22d-0acf-467d-b0a6-11877c9c9893 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:59.232342 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d901f22d-0acf-467d-b0a6-11877c9c9893 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:59.244219 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d901f22d-0acf-467d-b0a6-11877c9c9893 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:59.245678 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d901f22d-0acf-467d-b0a6-11877c9c9893 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:59.260299 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-d901f22d-0acf-467d-b0a6-11877c9c9893 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:33:59.260299 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:59.260299 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:59.260299 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:59.260299 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:59.260299 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:59.260299 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:59.260299 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:59.260299 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:59.260299 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:59.260299 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:59.260299 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:59.260299 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:33:59.260299 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:33:59.260299 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:33:59.260299 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:33:59.262495 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:59.262495 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:59.262495 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:59.262495 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:59.262495 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:59.262495 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:59.262495 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:59.262495 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:59.262495 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:59.262495 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:59.262495 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:59.262495 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:59.262495 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:59.262495 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:59.262495 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:59.262495 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:59.262495 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:33:59.263679 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:33:59.263679 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:33:59.263679 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:33:59.263679 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 306/612] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:33:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 10:33:59.269616 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eea46e8-4b35-4896-93e7-e40368d61306 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:59.341691 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:59.405215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eea46e8-4b35-4896-93e7-e40368d61306 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:59.446984 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff545fd8-8533-4124-a38b-7165f5222ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:59.447628 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff545fd8-8533-4124-a38b-7165f5222ad6 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:59.447913 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff545fd8-8533-4124-a38b-7165f5222ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:59.448188 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff545fd8-8533-4124-a38b-7165f5222ad6 None admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:59.448834 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff545fd8-8533-4124-a38b-7165f5222ad6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:59.449138 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff545fd8-8533-4124-a38b-7165f5222ad6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:59.450590 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff545fd8-8533-4124-a38b-7165f5222ad6 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:59.463198 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 307/613] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:33:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:59.468484 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d901f22d-0acf-467d-b0a6-11877c9c9893 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:59.595488 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d901f22d-0acf-467d-b0a6-11877c9c9893 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:59.617638 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:59.642747 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f105a52b-25ed-40be-8d10-948c05d55687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:59.643391 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f105a52b-25ed-40be-8d10-948c05d55687 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:59.643690 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f105a52b-25ed-40be-8d10-948c05d55687 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:59.643965 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f105a52b-25ed-40be-8d10-948c05d55687 None admin] PATH_INFO: `/v3/users/swift` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:59.654049 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f105a52b-25ed-40be-8d10-948c05d55687 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:59.655516 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f105a52b-25ed-40be-8d10-948c05d55687 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:59.669432 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-f105a52b-25ed-40be-8d10-948c05d55687 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 11 10:33:59.669432 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:33:59.669432 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:33:59.669432 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:33:59.669432 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:33:59.669432 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:33:59.669432 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:33:59.669432 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:33:59.669432 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:33:59.669432 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:33:59.669432 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:33:59.669432 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:33:59.669432 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 10:33:59.669432 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 10:33:59.669432 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 10:33:59.669432 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 10:33:59.671741 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:33:59.671741 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:33:59.671741 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 10:33:59.671741 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:33:59.671741 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 10:33:59.671741 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:33:59.671741 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:33:59.671741 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:33:59.671741 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:33:59.671741 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:33:59.671741 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:33:59.671741 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:33:59.671741 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:33:59.671741 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:33:59.671741 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:33:59.671741 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:33:59.671741 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:33:59.671741 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:33:59.673977 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:33:59.673977 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:33:59.673977 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 10:33:59.673977 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 10:33:59.673977 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 10:33:59.673977 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 10:33:59.673977 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 10:33:59.673977 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 10:33:59.673977 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 11 10:33:59.673977 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:33:59.673977 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 307/614] 10.222.0.15 () {56 vars in 1096 bytes} [Sun Sep 11 10:33:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:33:59.676773 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff545fd8-8533-4124-a38b-7165f5222ad6 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:33:59.805848 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff545fd8-8533-4124-a38b-7165f5222ad6 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:33:59.817008 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:33:59.853622 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e45a7b-6a37-4c3e-ad74-d2833c974c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:33:59.854160 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e45a7b-6a37-4c3e-ad74-d2833c974c39 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:33:59.854442 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e45a7b-6a37-4c3e-ad74-d2833c974c39 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:33:59.854715 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e45a7b-6a37-4c3e-ad74-d2833c974c39 None admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:33:59.855459 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e45a7b-6a37-4c3e-ad74-d2833c974c39 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:33:59.855773 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e45a7b-6a37-4c3e-ad74-d2833c974c39 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:33:59.857197 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e45a7b-6a37-4c3e-ad74-d2833c974c39 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:33:59.875133 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 308/615] 10.222.0.15 () {56 vars in 1135 bytes} [Sun Sep 11 10:33:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:33:59.880783 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f105a52b-25ed-40be-8d10-948c05d55687 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:00.056540 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f105a52b-25ed-40be-8d10-948c05d55687 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:00.102574 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1479ddca-8f42-4ee2-adb6-dc98e7248786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:00.103216 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1479ddca-8f42-4ee2-adb6-dc98e7248786 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:00.103516 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1479ddca-8f42-4ee2-adb6-dc98e7248786 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:00.103834 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1479ddca-8f42-4ee2-adb6-dc98e7248786 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:00.119135 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1479ddca-8f42-4ee2-adb6-dc98e7248786 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:00.120619 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1479ddca-8f42-4ee2-adb6-dc98e7248786 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:00.138230 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-1479ddca-8f42-4ee2-adb6-dc98e7248786 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:00.138230 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:00.138230 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:00.138230 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:00.138230 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:00.138230 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:00.138230 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:00.138230 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:00.138230 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:00.138230 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:00.138230 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:00.138230 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:00.138230 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:34:00.138230 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:34:00.138230 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:34:00.138230 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:34:00.140472 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:00.140472 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:00.140472 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:00.140472 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:00.140472 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:00.140472 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:00.140472 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:00.140472 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:00.140472 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:00.140472 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:00.140472 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:00.140472 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:00.140472 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:00.140472 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:00.140472 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:00.140472 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:00.140472 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:34:00.142587 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:34:00.142587 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:00.142587 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:00.143065 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 308/616] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:33:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:00.149213 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e45a7b-6a37-4c3e-ad74-d2833c974c39 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:00.202581 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff26be36-114b-426b-9bf5-e2ec645df193 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:00.202581 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff26be36-114b-426b-9bf5-e2ec645df193 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:00.202581 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff26be36-114b-426b-9bf5-e2ec645df193 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:00.203463 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 309/617] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:34:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:34:00.208834 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68318cc9-22a7-4729-8516-18139630fbe1 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:00.209366 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68318cc9-22a7-4729-8516-18139630fbe1 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:00.209844 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68318cc9-22a7-4729-8516-18139630fbe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:00.230402 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-68318cc9-22a7-4729-8516-18139630fbe1 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:34:00.232586 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-68318cc9-22a7-4729-8516-18139630fbe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:34:00.273254 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:00.292489 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e45a7b-6a37-4c3e-ad74-d2833c974c39 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:00.338871 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a640d5-14f5-4d63-9353-21d223efedcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:00.339636 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a640d5-14f5-4d63-9353-21d223efedcf None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:00.339947 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a640d5-14f5-4d63-9353-21d223efedcf None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:00.340302 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a640d5-14f5-4d63-9353-21d223efedcf None admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:00.340950 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a640d5-14f5-4d63-9353-21d223efedcf None admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:00.341258 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a640d5-14f5-4d63-9353-21d223efedcf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:00.342630 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a640d5-14f5-4d63-9353-21d223efedcf None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:00.359644 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 310/618] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:34:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 10:34:00.362494 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 309/619] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:34:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:00.368201 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68318cc9-22a7-4729-8516-18139630fbe1 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:00.514051 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68318cc9-22a7-4729-8516-18139630fbe1 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:00.561021 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8186e57c-19ed-451e-9802-1046d7a8aaee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:00.561658 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8186e57c-19ed-451e-9802-1046d7a8aaee None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:00.561940 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8186e57c-19ed-451e-9802-1046d7a8aaee None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:00.562199 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8186e57c-19ed-451e-9802-1046d7a8aaee None admin] PATH_INFO: `/v3/projects/service` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:00.572458 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8186e57c-19ed-451e-9802-1046d7a8aaee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:00.573978 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8186e57c-19ed-451e-9802-1046d7a8aaee None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:00.589922 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-8186e57c-19ed-451e-9802-1046d7a8aaee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:34:00.589922 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:00.589922 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:00.589922 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:00.589922 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:00.589922 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:00.589922 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:00.589922 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:00.589922 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:00.589922 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:00.589922 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:00.589922 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:00.589922 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 10:34:00.589922 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 10:34:00.589922 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 10:34:00.590856 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 10:34:00.590856 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:00.590856 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:00.590856 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:00.590856 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:00.590856 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:00.590856 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:00.590856 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:00.590856 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:00.590856 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:00.590856 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:00.590856 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:00.590856 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:00.590856 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:00.590856 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:00.590856 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:00.590856 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:00.591766 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 10:34:00.591766 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 10:34:00.591766 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 10:34:00.591766 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 10:34:00.591766 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 10:34:00.591766 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 10:34:00.591766 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:34:00.591766 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:00.591766 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 311/620] 10.222.0.15 () {56 vars in 1111 bytes} [Sun Sep 11 10:34:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:00.596663 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a640d5-14f5-4d63-9353-21d223efedcf None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:00.637356 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:00.735099 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a640d5-14f5-4d63-9353-21d223efedcf None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:00.778331 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb63fdee-776b-4840-8da2-b397fef8a859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:00.778947 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb63fdee-776b-4840-8da2-b397fef8a859 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:00.779230 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb63fdee-776b-4840-8da2-b397fef8a859 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:00.779536 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb63fdee-776b-4840-8da2-b397fef8a859 None admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:00.780231 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb63fdee-776b-4840-8da2-b397fef8a859 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:00.780537 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb63fdee-776b-4840-8da2-b397fef8a859 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:00.781988 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb63fdee-776b-4840-8da2-b397fef8a859 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:00.802738 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 310/621] 10.222.0.15 () {56 vars in 1148 bytes} [Sun Sep 11 10:34:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:00.808442 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8186e57c-19ed-451e-9802-1046d7a8aaee None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:00.830603 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:00.940283 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb63fdee-776b-4840-8da2-b397fef8a859 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:00.941813 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8186e57c-19ed-451e-9802-1046d7a8aaee None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:00.991137 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68774363-0e60-41e1-81c4-aa1337dd8cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:00.997549 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68774363-0e60-41e1-81c4-aa1337dd8cdb None admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:00.997845 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68774363-0e60-41e1-81c4-aa1337dd8cdb None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:00.998124 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68774363-0e60-41e1-81c4-aa1337dd8cdb None admin] PATH_INFO: `/v3/projects/5e1f845f99f04202b83621cb65e0adc2/users/cc7ca566e66a45b29e3d02f47bf84981/roles/3cb7ebbc707d453586b3d0f4ab3cf103` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:01.026188 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68774363-0e60-41e1-81c4-aa1337dd8cdb None admin] RBAC: Authorizing `identity:create_grant(project_id=5e1f845f99f04202b83621cb65e0adc2, user_id=cc7ca566e66a45b29e3d02f47bf84981, role_id=3cb7ebbc707d453586b3d0f4ab3cf103)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:01.027905 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68774363-0e60-41e1-81c4-aa1337dd8cdb None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:01.050865 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 312/622] 10.222.0.15 () {58 vars in 1439 bytes} [Sun Sep 11 10:34:00 2022] PUT /identity/v3/projects/5e1f845f99f04202b83621cb65e0adc2/users/cc7ca566e66a45b29e3d02f47bf84981/roles/3cb7ebbc707d453586b3d0f4ab3cf103 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:34:01.084162 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:01.102529 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb63fdee-776b-4840-8da2-b397fef8a859 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:01.141226 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2565c060-c08a-4903-a6ee-c11ad8dfe34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:01.141950 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2565c060-c08a-4903-a6ee-c11ad8dfe34a None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:01.142232 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2565c060-c08a-4903-a6ee-c11ad8dfe34a None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:01.142503 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2565c060-c08a-4903-a6ee-c11ad8dfe34a None admin] PATH_INFO: `/v3/roles/service` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:01.155563 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2565c060-c08a-4903-a6ee-c11ad8dfe34a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:34:01.160595 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-2565c060-c08a-4903-a6ee-c11ad8dfe34a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 10:34:01.160595 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:01.160595 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:01.160595 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:01.160595 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:01.160595 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:01.160595 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:01.160595 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:01.160595 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:01.160595 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:01.160595 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:01.160595 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:01.160595 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:34:01.160595 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:34:01.160595 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:34:01.160595 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:34:01.161622 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:34:01.161622 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:34:01.161622 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:34:01.161622 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:34:01.161622 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:01.161622 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:01.161622 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:01.161622 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:01.161622 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:01.161622 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:01.161622 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:01.161622 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:01.161622 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:01.161622 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:01.161622 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:01.161622 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:01.162465 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:01.162465 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:01.162465 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:01.162465 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:01.162465 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:34:01.162465 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:34:01.162465 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:34:01.162465 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:34:01.162465 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:34:01.162465 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:34:01.162465 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 10:34:01.162465 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:01.162465 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 311/623] 10.222.0.15 () {56 vars in 1102 bytes} [Sun Sep 11 10:34:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 10:34:01.168025 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68774363-0e60-41e1-81c4-aa1337dd8cdb None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:01.258913 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:01.313011 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68774363-0e60-41e1-81c4-aa1337dd8cdb None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:01.347243 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d106eb-6c03-4f0c-b555-8115c447da19 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:01.347896 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d106eb-6c03-4f0c-b555-8115c447da19 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:01.347896 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d106eb-6c03-4f0c-b555-8115c447da19 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:01.349316 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 312/624] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:34:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 10:34:01.355672 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4cc544-c288-4cf9-9701-a2ec44d5b2ee None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:01.356148 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4cc544-c288-4cf9-9701-a2ec44d5b2ee None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:01.356582 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4cc544-c288-4cf9-9701-a2ec44d5b2ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:01.357295 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60fda8e-3abf-4b4a-aad7-71b464467f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:01.358105 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60fda8e-3abf-4b4a-aad7-71b464467f2b None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:01.358555 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60fda8e-3abf-4b4a-aad7-71b464467f2b None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:01.358985 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60fda8e-3abf-4b4a-aad7-71b464467f2b None admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:01.359857 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60fda8e-3abf-4b4a-aad7-71b464467f2b None admin] RBAC: Adding query filter params (name=service) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:01.360367 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60fda8e-3abf-4b4a-aad7-71b464467f2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:01.361902 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60fda8e-3abf-4b4a-aad7-71b464467f2b None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:01.375951 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 313/625] 10.222.0.15 () {56 vars in 1103 bytes} [Sun Sep 11 10:34:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:01.381236 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60fda8e-3abf-4b4a-aad7-71b464467f2b None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:01.382604 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-2f4cc544-c288-4cf9-9701-a2ec44d5b2ee None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:34:01.384039 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-2f4cc544-c288-4cf9-9701-a2ec44d5b2ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:34:01.473219 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 313/626] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:34:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:34:01.473219 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c77aa2a-f73f-48a3-aadf-7561b953e741 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:01.473219 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c77aa2a-f73f-48a3-aadf-7561b953e741 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:01.473219 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c77aa2a-f73f-48a3-aadf-7561b953e741 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:01.473219 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 314/627] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:34:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:34:01.480195 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e574ec38-e444-4ea5-b9b0-c98bf4b3b948 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:01.480482 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e574ec38-e444-4ea5-b9b0-c98bf4b3b948 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:01.480823 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e574ec38-e444-4ea5-b9b0-c98bf4b3b948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:01.500688 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-e574ec38-e444-4ea5-b9b0-c98bf4b3b948 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:34:01.504357 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-e574ec38-e444-4ea5-b9b0-c98bf4b3b948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:34:01.553646 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60fda8e-3abf-4b4a-aad7-71b464467f2b None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:01.603309 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 315/628] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:34:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:34:01.610618 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4542c5eb-17c7-4044-9679-5053630ea882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:01.611340 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4542c5eb-17c7-4044-9679-5053630ea882 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:01.611708 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4542c5eb-17c7-4044-9679-5053630ea882 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:01.611985 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4542c5eb-17c7-4044-9679-5053630ea882 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:01.623823 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:01.628627 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4542c5eb-17c7-4044-9679-5053630ea882 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:01.630139 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4542c5eb-17c7-4044-9679-5053630ea882 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:01.651914 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-4542c5eb-17c7-4044-9679-5053630ea882 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:01.651914 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:01.651914 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:01.651914 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:01.651914 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:01.651914 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:01.651914 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:01.651914 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:01.651914 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:01.651914 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:01.651914 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:01.651914 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:01.651914 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:34:01.651914 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:34:01.651914 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:34:01.651914 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:34:01.652981 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:01.652981 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:01.652981 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:01.652981 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:01.652981 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:01.652981 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:01.652981 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:01.652981 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:01.652981 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:01.652981 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:01.652981 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:01.652981 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:01.652981 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:01.652981 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:01.652981 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:01.652981 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:01.652981 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:34:01.653880 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:34:01.653880 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:01.653880 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:01.658070 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 314/629] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:34:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:01.659597 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e574ec38-e444-4ea5-b9b0-c98bf4b3b948 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:01.672887 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:01.766731 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e574ec38-e444-4ea5-b9b0-c98bf4b3b948 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:01.774931 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6dcccc-eaa8-4353-af0f-5be76b672f8f None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:01.775205 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6dcccc-eaa8-4353-af0f-5be76b672f8f None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:01.775500 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6dcccc-eaa8-4353-af0f-5be76b672f8f None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:01.776979 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 315/630] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:34:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 10:34:01.781734 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061cfdee-aed0-482c-b56f-9020c9492ce7 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:01.782012 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061cfdee-aed0-482c-b56f-9020c9492ce7 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:01.782282 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061cfdee-aed0-482c-b56f-9020c9492ce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:01.812404 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-061cfdee-aed0-482c-b56f-9020c9492ce7 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:34:01.813503 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-061cfdee-aed0-482c-b56f-9020c9492ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:34:01.836588 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b435d3-9df0-40bd-b106-60b28d58eb81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:01.837200 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b435d3-9df0-40bd-b106-60b28d58eb81 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:01.837478 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b435d3-9df0-40bd-b106-60b28d58eb81 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:01.837744 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b435d3-9df0-40bd-b106-60b28d58eb81 None admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:01.838635 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b435d3-9df0-40bd-b106-60b28d58eb81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:01.838635 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b435d3-9df0-40bd-b106-60b28d58eb81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:01.840090 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b435d3-9df0-40bd-b106-60b28d58eb81 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:01.863447 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 316/631] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:34:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:01.863447 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b435d3-9df0-40bd-b106-60b28d58eb81 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:01.922316 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 316/632] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:34:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 11 10:34:01.923151 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061cfdee-aed0-482c-b56f-9020c9492ce7 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:01.934175 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:01.971060 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b435d3-9df0-40bd-b106-60b28d58eb81 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:02.008248 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0ce804-719a-4cea-b7d4-3fdbe18eebb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:02.008891 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0ce804-719a-4cea-b7d4-3fdbe18eebb3 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:02.009174 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0ce804-719a-4cea-b7d4-3fdbe18eebb3 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:02.009447 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0ce804-719a-4cea-b7d4-3fdbe18eebb3 None admin] PATH_INFO: `/v3/users/glance` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:02.017987 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0ce804-719a-4cea-b7d4-3fdbe18eebb3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:02.019607 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0ce804-719a-4cea-b7d4-3fdbe18eebb3 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:02.032219 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-2b0ce804-719a-4cea-b7d4-3fdbe18eebb3 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 11 10:34:02.032219 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:02.032219 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:02.032219 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:02.032219 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:02.032219 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:02.032219 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:02.032219 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:02.032219 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:02.032219 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:02.032219 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:02.032219 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:02.032219 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 10:34:02.032219 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 10:34:02.032219 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 10:34:02.032219 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 10:34:02.033212 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:02.033212 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:02.033212 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 10:34:02.033212 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:34:02.033212 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 10:34:02.033212 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:34:02.033212 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:02.033212 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:02.033212 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:02.033212 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:02.033212 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:02.033212 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:02.033212 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:02.033212 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:02.033212 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:02.033212 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:02.033212 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:02.033212 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:02.034170 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:02.034170 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:02.034170 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 10:34:02.034170 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 10:34:02.034170 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 10:34:02.034170 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 10:34:02.034170 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 10:34:02.034170 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 10:34:02.034170 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 11 10:34:02.034170 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:02.038157 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 317/633] 10.222.0.15 () {56 vars in 1099 bytes} [Sun Sep 11 10:34:01 2022] GET /identity/v3/users/glance => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 10:34:02.039116 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0ce804-719a-4cea-b7d4-3fdbe18eebb3 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:02.054492 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061cfdee-aed0-482c-b56f-9020c9492ce7 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:02.074445 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:02.106528 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119e3751-89c3-4530-8df5-5f6ae297d1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:02.107714 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119e3751-89c3-4530-8df5-5f6ae297d1d8 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:02.108148 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119e3751-89c3-4530-8df5-5f6ae297d1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:02.108859 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119e3751-89c3-4530-8df5-5f6ae297d1d8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:02.123338 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119e3751-89c3-4530-8df5-5f6ae297d1d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:34:02.130442 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-119e3751-89c3-4530-8df5-5f6ae297d1d8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 10:34:02.130442 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:02.130442 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:02.130442 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:02.130442 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:02.130442 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:02.130442 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:02.130442 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:02.130442 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:02.130442 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:02.130442 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:02.130442 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:02.130442 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:34:02.130442 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:34:02.130442 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:34:02.130442 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:34:02.131565 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:34:02.131565 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:34:02.131565 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:34:02.131565 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:34:02.131565 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:02.131565 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:02.131565 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:02.131565 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:02.131565 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:02.131565 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:02.131565 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:02.131565 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:02.131565 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:02.131565 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:02.131565 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:02.131565 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:02.132910 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:02.132910 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:02.132910 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:02.132910 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:02.132910 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:34:02.132910 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:34:02.132910 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:34:02.132910 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:34:02.132910 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:34:02.132910 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:34:02.132910 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 10:34:02.132910 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:02.135567 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 317/634] 10.222.0.15 () {56 vars in 1102 bytes} [Sun Sep 11 10:34:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:02.136584 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119e3751-89c3-4530-8df5-5f6ae297d1d8 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:02.157529 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0ce804-719a-4cea-b7d4-3fdbe18eebb3 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:02.185846 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:02.195196 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f33e2d-42d3-4148-8c04-e6fea4b971e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:02.195869 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f33e2d-42d3-4148-8c04-e6fea4b971e3 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:02.196164 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f33e2d-42d3-4148-8c04-e6fea4b971e3 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:02.196430 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f33e2d-42d3-4148-8c04-e6fea4b971e3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:02.207362 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f33e2d-42d3-4148-8c04-e6fea4b971e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:34:02.213315 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-a4f33e2d-42d3-4148-8c04-e6fea4b971e3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 10:34:02.213315 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:02.213315 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:02.213315 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:02.213315 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:02.213315 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:02.213315 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:02.213315 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:02.213315 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:02.213315 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:02.213315 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:02.213315 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:02.213315 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:34:02.213315 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:34:02.213315 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:34:02.213315 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:34:02.214286 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:34:02.214286 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:34:02.214286 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:34:02.214286 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:34:02.214286 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:02.214286 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:02.214286 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:02.214286 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:02.214286 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:02.214286 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:02.214286 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:02.214286 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:02.214286 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:02.214286 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:02.214286 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:02.214286 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:02.215723 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:02.215723 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:02.215723 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:02.215723 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:02.215723 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:34:02.215723 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:34:02.215723 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:34:02.215723 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:34:02.215723 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:34:02.215723 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:34:02.215723 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 10:34:02.215723 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:02.215723 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 318/635] 10.222.0.15 () {56 vars in 1102 bytes} [Sun Sep 11 10:34:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:02.215723 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f33e2d-42d3-4148-8c04-e6fea4b971e3 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:02.264842 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119e3751-89c3-4530-8df5-5f6ae297d1d8 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:02.305942 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0d81ba-0b73-49fd-a788-01ed8f59bd1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:02.306425 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d81ba-0b73-49fd-a788-01ed8f59bd1d None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:02.306844 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d81ba-0b73-49fd-a788-01ed8f59bd1d None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:02.307129 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d81ba-0b73-49fd-a788-01ed8f59bd1d None admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:02.307879 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0d81ba-0b73-49fd-a788-01ed8f59bd1d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:02.308186 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0d81ba-0b73-49fd-a788-01ed8f59bd1d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:02.309642 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0d81ba-0b73-49fd-a788-01ed8f59bd1d None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:02.331544 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f33e2d-42d3-4148-8c04-e6fea4b971e3 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:02.332830 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 318/636] 10.222.0.15 () {56 vars in 1137 bytes} [Sun Sep 11 10:34:02 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:02.333483 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0d81ba-0b73-49fd-a788-01ed8f59bd1d None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:02.365716 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:02.368020 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe813cf-b709-4d1e-9b03-ba441208ec94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:02.369180 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe813cf-b709-4d1e-9b03-ba441208ec94 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:02.369460 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe813cf-b709-4d1e-9b03-ba441208ec94 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:02.369777 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe813cf-b709-4d1e-9b03-ba441208ec94 None admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:02.370607 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe813cf-b709-4d1e-9b03-ba441208ec94 None admin] RBAC: Adding query filter params (name=service) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:02.370919 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe813cf-b709-4d1e-9b03-ba441208ec94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:02.373135 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe813cf-b709-4d1e-9b03-ba441208ec94 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:02.382991 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 319/637] 10.222.0.15 () {56 vars in 1103 bytes} [Sun Sep 11 10:34:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:02.382991 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe813cf-b709-4d1e-9b03-ba441208ec94 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:02.435510 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:02.444605 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0d81ba-0b73-49fd-a788-01ed8f59bd1d None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:02.481523 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808fc328-5ef7-4844-85c0-33bfa0f051b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:02.482123 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808fc328-5ef7-4844-85c0-33bfa0f051b0 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:02.482396 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808fc328-5ef7-4844-85c0-33bfa0f051b0 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:02.482665 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808fc328-5ef7-4844-85c0-33bfa0f051b0 None admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:02.483314 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808fc328-5ef7-4844-85c0-33bfa0f051b0 None admin] RBAC: Adding query filter params (name=service) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:02.483643 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808fc328-5ef7-4844-85c0-33bfa0f051b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:02.485001 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808fc328-5ef7-4844-85c0-33bfa0f051b0 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:02.489771 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe813cf-b709-4d1e-9b03-ba441208ec94 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:02.498043 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 319/638] 10.222.0.15 () {56 vars in 1103 bytes} [Sun Sep 11 10:34:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:02.498043 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808fc328-5ef7-4844-85c0-33bfa0f051b0 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:02.528421 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae632558-0b18-427d-9c40-c521f066f9c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:02.529218 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae632558-0b18-427d-9c40-c521f066f9c5 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:02.529829 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae632558-0b18-427d-9c40-c521f066f9c5 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:02.530243 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae632558-0b18-427d-9c40-c521f066f9c5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:02.542185 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae632558-0b18-427d-9c40-c521f066f9c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:34:02.548459 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-ae632558-0b18-427d-9c40-c521f066f9c5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 10:34:02.548459 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:02.548459 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:02.548459 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:02.548459 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:02.548459 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:02.548459 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:02.548459 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:02.548459 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:02.548459 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:02.548459 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:02.548459 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:02.548459 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:34:02.548459 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:34:02.548459 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:34:02.548459 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:34:02.549400 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:34:02.549400 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:34:02.549400 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:34:02.549400 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:34:02.549400 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:02.549400 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:02.549400 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:02.549400 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:02.549400 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:02.549400 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:02.549400 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:02.549400 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:02.549400 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:02.549400 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:02.549400 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:02.549400 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:02.552623 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:02.552623 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:02.552623 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:02.552623 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:02.552623 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:34:02.552623 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:34:02.552623 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:34:02.552623 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:34:02.552623 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:34:02.552623 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:34:02.552623 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 10:34:02.552623 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:02.552623 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 320/639] 10.222.0.15 () {56 vars in 1102 bytes} [Sun Sep 11 10:34:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:02.552623 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae632558-0b18-427d-9c40-c521f066f9c5 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:02.594102 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:02.603994 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808fc328-5ef7-4844-85c0-33bfa0f051b0 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:02.638666 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a070f1-1dce-476a-9c66-96c9f98536c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:02.639279 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a070f1-1dce-476a-9c66-96c9f98536c0 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:02.639599 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a070f1-1dce-476a-9c66-96c9f98536c0 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:02.639875 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a070f1-1dce-476a-9c66-96c9f98536c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:02.671039 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae632558-0b18-427d-9c40-c521f066f9c5 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:02.685600 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:02.712285 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a070f1-1dce-476a-9c66-96c9f98536c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:02.713836 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a070f1-1dce-476a-9c66-96c9f98536c0 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:02.714868 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3be7a57-d8a7-4ffd-b64d-1b02d8d8900e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:02.715574 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be7a57-d8a7-4ffd-b64d-1b02d8d8900e None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:02.715797 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be7a57-d8a7-4ffd-b64d-1b02d8d8900e None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:02.716065 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be7a57-d8a7-4ffd-b64d-1b02d8d8900e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:02.727336 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3be7a57-d8a7-4ffd-b64d-1b02d8d8900e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:02.729111 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3be7a57-d8a7-4ffd-b64d-1b02d8d8900e None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:02.744384 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-d3be7a57-d8a7-4ffd-b64d-1b02d8d8900e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:02.744384 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:02.744384 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:02.744384 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:02.744384 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:02.744384 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:02.744384 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:02.744384 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:02.744384 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:02.744384 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:02.744384 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:02.744384 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:02.744384 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:34:02.744384 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:34:02.744384 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:34:02.744384 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:34:02.745338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:02.745338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:02.745338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:02.745338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:02.745338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:02.745338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:02.745338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:02.745338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:02.745338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:02.745338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:02.745338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:02.745338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:02.745338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:02.745338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:02.745338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:02.745338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:02.745338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:34:02.746257 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:34:02.746257 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:02.746257 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:02.746257 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 321/640] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:34:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:02.746257 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3be7a57-d8a7-4ffd-b64d-1b02d8d8900e None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:02.813133 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:02.827148 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 320/641] 10.222.0.15 () {58 vars in 1282 bytes} [Sun Sep 11 10:34:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 10:34:02.827148 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a070f1-1dce-476a-9c66-96c9f98536c0 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:02.854409 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3be7a57-d8a7-4ffd-b64d-1b02d8d8900e None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:02.889331 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b394150b-c669-4bf8-a572-6dd01f617c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:02.889952 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b394150b-c669-4bf8-a572-6dd01f617c52 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:02.890278 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b394150b-c669-4bf8-a572-6dd01f617c52 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:02.890492 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b394150b-c669-4bf8-a572-6dd01f617c52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:02.901634 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b394150b-c669-4bf8-a572-6dd01f617c52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:02.903076 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b394150b-c669-4bf8-a572-6dd01f617c52 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:02.917247 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-b394150b-c669-4bf8-a572-6dd01f617c52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:02.917247 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:02.917247 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:02.917247 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:02.917247 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:02.917247 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:02.917247 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:02.917247 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:02.917247 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:02.917247 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:02.917247 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:02.917247 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:02.917247 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:34:02.917247 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:34:02.917247 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:34:02.917247 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:34:02.918273 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:02.918273 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:02.918273 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:02.918273 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:02.918273 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:02.918273 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:02.918273 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:02.918273 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:02.918273 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:02.918273 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:02.918273 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:02.918273 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:02.918273 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:02.918273 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:02.918273 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:02.918273 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:02.918273 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:34:02.919219 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:34:02.919219 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:02.919219 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:02.919219 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 322/642] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:34:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:02.919219 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b394150b-c669-4bf8-a572-6dd01f617c52 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:02.930661 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a070f1-1dce-476a-9c66-96c9f98536c0 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:02.938001 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:02.969351 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daef62d-11a9-40b1-8d5e-f6c74620adc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:02.970274 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daef62d-11a9-40b1-8d5e-f6c74620adc4 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:02.970748 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daef62d-11a9-40b1-8d5e-f6c74620adc4 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:02.971025 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daef62d-11a9-40b1-8d5e-f6c74620adc4 None admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:02.971709 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daef62d-11a9-40b1-8d5e-f6c74620adc4 None admin] RBAC: Adding query filter params (name=service) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:02.972015 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daef62d-11a9-40b1-8d5e-f6c74620adc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:02.973473 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daef62d-11a9-40b1-8d5e-f6c74620adc4 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:02.982336 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 321/643] 10.222.0.15 () {56 vars in 1103 bytes} [Sun Sep 11 10:34:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:02.983134 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daef62d-11a9-40b1-8d5e-f6c74620adc4 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:03.038588 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b394150b-c669-4bf8-a572-6dd01f617c52 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:03.040377 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:03.075245 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f21d69-f0dc-4f8e-990b-60d5fd44ad99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:03.075878 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f21d69-f0dc-4f8e-990b-60d5fd44ad99 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:03.076429 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f21d69-f0dc-4f8e-990b-60d5fd44ad99 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:03.076429 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f21d69-f0dc-4f8e-990b-60d5fd44ad99 None admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:03.077031 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f21d69-f0dc-4f8e-990b-60d5fd44ad99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:03.077370 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f21d69-f0dc-4f8e-990b-60d5fd44ad99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:03.078692 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f21d69-f0dc-4f8e-990b-60d5fd44ad99 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:03.089595 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daef62d-11a9-40b1-8d5e-f6c74620adc4 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:03.092214 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 323/644] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:34:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:03.093132 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f21d69-f0dc-4f8e-990b-60d5fd44ad99 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:03.128893 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48574fd-acb6-488e-a57e-27641e505bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:03.129429 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48574fd-acb6-488e-a57e-27641e505bc9 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:03.129708 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48574fd-acb6-488e-a57e-27641e505bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:03.129977 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48574fd-acb6-488e-a57e-27641e505bc9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:03.141804 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48574fd-acb6-488e-a57e-27641e505bc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:03.143310 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48574fd-acb6-488e-a57e-27641e505bc9 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:03.157824 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-e48574fd-acb6-488e-a57e-27641e505bc9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:03.157824 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:03.157824 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:03.157824 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:03.157824 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:03.157824 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:03.157824 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:03.157824 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:03.157824 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:03.157824 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:03.157824 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:03.157824 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:03.157824 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:34:03.157824 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:34:03.157824 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:34:03.157824 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:34:03.159001 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:03.159001 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:03.159001 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:03.159001 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:03.159001 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:03.159001 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:03.159001 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:03.159001 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:03.159001 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:03.159001 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:03.159001 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:03.159001 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:03.159001 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:03.159001 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:03.159001 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:03.159001 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:03.159001 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:34:03.160290 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:34:03.160290 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:03.160290 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:03.160290 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 322/645] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:34:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:03.160290 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48574fd-acb6-488e-a57e-27641e505bc9 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:03.181833 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:03.210514 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f21d69-f0dc-4f8e-990b-60d5fd44ad99 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:03.250052 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf3bbd4-6fb5-44e4-ad7b-31d68383443b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:03.250659 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf3bbd4-6fb5-44e4-ad7b-31d68383443b None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:03.250934 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf3bbd4-6fb5-44e4-ad7b-31d68383443b None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:03.251206 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf3bbd4-6fb5-44e4-ad7b-31d68383443b None admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:03.251862 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf3bbd4-6fb5-44e4-ad7b-31d68383443b None admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:03.252152 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf3bbd4-6fb5-44e4-ad7b-31d68383443b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:03.253496 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf3bbd4-6fb5-44e4-ad7b-31d68383443b None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:03.265694 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 324/646] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:34:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:03.266495 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf3bbd4-6fb5-44e4-ad7b-31d68383443b None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:03.272936 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48574fd-acb6-488e-a57e-27641e505bc9 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:03.280348 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:03.314628 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c849d05d-9279-409b-bcf8-bbff1d08e163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:03.315534 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c849d05d-9279-409b-bcf8-bbff1d08e163 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:03.315886 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c849d05d-9279-409b-bcf8-bbff1d08e163 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:03.316161 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c849d05d-9279-409b-bcf8-bbff1d08e163 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:03.326699 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c849d05d-9279-409b-bcf8-bbff1d08e163 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:03.328174 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c849d05d-9279-409b-bcf8-bbff1d08e163 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:03.343306 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-c849d05d-9279-409b-bcf8-bbff1d08e163 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:03.343306 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:03.343306 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:03.343306 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:03.343306 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:03.343306 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:03.343306 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:03.343306 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:03.343306 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:03.343306 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:03.343306 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:03.343306 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:03.343306 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:34:03.343306 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:34:03.343306 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:34:03.343306 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:34:03.344306 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:03.344306 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:03.344306 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:03.344306 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:03.344306 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:03.344306 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:03.344306 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:03.344306 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:03.344306 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:03.344306 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:03.344306 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:03.344306 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:03.344306 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:03.344306 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:03.344306 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:03.344306 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:03.344306 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:34:03.345202 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:34:03.345202 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:03.345202 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:03.345779 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 323/647] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:34:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:03.346486 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c849d05d-9279-409b-bcf8-bbff1d08e163 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:03.381244 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf3bbd4-6fb5-44e4-ad7b-31d68383443b None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:03.417978 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d104b03-1823-419c-9cba-cab14d11b058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:03.418606 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d104b03-1823-419c-9cba-cab14d11b058 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:03.418929 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d104b03-1823-419c-9cba-cab14d11b058 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:03.419164 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d104b03-1823-419c-9cba-cab14d11b058 None admin] PATH_INFO: `/v3/users/nova` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:03.421323 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:03.428577 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d104b03-1823-419c-9cba-cab14d11b058 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:03.431643 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d104b03-1823-419c-9cba-cab14d11b058 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:03.447000 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-2d104b03-1823-419c-9cba-cab14d11b058 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 11 10:34:03.447000 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:03.447000 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:03.447000 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:03.447000 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:03.447000 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:03.447000 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:03.447000 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:03.447000 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:03.447000 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:03.447000 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:03.447000 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:03.447000 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 10:34:03.447000 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 10:34:03.447000 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 10:34:03.447000 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 10:34:03.448073 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:03.448073 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:03.448073 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 10:34:03.448073 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:34:03.448073 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 10:34:03.448073 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:34:03.448073 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:03.448073 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:03.448073 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:03.448073 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:03.448073 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:03.448073 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:03.448073 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:03.448073 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:03.448073 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:03.448073 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:03.448073 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:03.448073 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:03.449008 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:03.449008 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:03.449008 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 10:34:03.449008 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 10:34:03.449008 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 10:34:03.449008 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 10:34:03.449008 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 10:34:03.449008 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 10:34:03.449008 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 11 10:34:03.449008 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:03.449008 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 325/648] 10.222.0.15 () {56 vars in 1093 bytes} [Sun Sep 11 10:34:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:03.449600 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d104b03-1823-419c-9cba-cab14d11b058 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:03.459354 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c849d05d-9279-409b-bcf8-bbff1d08e163 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:03.494587 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5b888d-5a38-44d5-b767-091606dd7177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:03.495189 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5b888d-5a38-44d5-b767-091606dd7177 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:03.495492 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5b888d-5a38-44d5-b767-091606dd7177 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:03.495765 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5b888d-5a38-44d5-b767-091606dd7177 None admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:03.496393 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5b888d-5a38-44d5-b767-091606dd7177 None admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:03.496690 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5b888d-5a38-44d5-b767-091606dd7177 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:03.498059 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5b888d-5a38-44d5-b767-091606dd7177 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:03.509132 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 324/649] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:34:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:03.509928 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5b888d-5a38-44d5-b767-091606dd7177 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:03.530362 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:03.567660 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d104b03-1823-419c-9cba-cab14d11b058 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:03.603051 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f77756a-37dc-4616-84f1-1f89cdddf369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:03.603711 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f77756a-37dc-4616-84f1-1f89cdddf369 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:03.603993 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f77756a-37dc-4616-84f1-1f89cdddf369 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:03.604265 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f77756a-37dc-4616-84f1-1f89cdddf369 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:03.609710 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5b888d-5a38-44d5-b767-091606dd7177 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:03.611656 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f77756a-37dc-4616-84f1-1f89cdddf369 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:03.613074 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f77756a-37dc-4616-84f1-1f89cdddf369 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:03.625100 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-6f77756a-37dc-4616-84f1-1f89cdddf369 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 11 10:34:03.625100 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:03.625100 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:03.625100 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:03.625100 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:03.625100 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:03.625100 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:03.625100 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:03.625100 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:03.625100 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:03.625100 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:03.625100 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:03.625100 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 10:34:03.625100 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 10:34:03.625100 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 10:34:03.625100 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 10:34:03.626092 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:03.626092 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:03.626092 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 10:34:03.626092 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:34:03.626092 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 10:34:03.626092 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:34:03.626092 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:03.626092 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:03.626092 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:03.626092 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:03.626092 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:03.626092 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:03.626092 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:03.626092 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:03.626092 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:03.626092 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:03.626092 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:03.626092 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:03.627056 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:03.627056 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:03.627056 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 10:34:03.627056 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 10:34:03.627056 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 10:34:03.627056 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 10:34:03.627056 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 10:34:03.627056 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 10:34:03.627056 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 11 10:34:03.627056 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:03.627056 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 326/650] 10.222.0.15 () {56 vars in 1102 bytes} [Sun Sep 11 10:34:03 2022] GET /identity/v3/users/neutron => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:03.627056 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f77756a-37dc-4616-84f1-1f89cdddf369 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:03.650608 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacae074-115a-4e5b-88d6-6e5ff9998838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:03.651153 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacae074-115a-4e5b-88d6-6e5ff9998838 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:03.651452 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacae074-115a-4e5b-88d6-6e5ff9998838 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:03.651738 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacae074-115a-4e5b-88d6-6e5ff9998838 None admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:03.652366 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacae074-115a-4e5b-88d6-6e5ff9998838 None admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:03.652668 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacae074-115a-4e5b-88d6-6e5ff9998838 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:03.654143 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacae074-115a-4e5b-88d6-6e5ff9998838 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:03.666335 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 325/651] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:34:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:03.667124 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacae074-115a-4e5b-88d6-6e5ff9998838 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:03.682699 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:03.729224 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f77756a-37dc-4616-84f1-1f89cdddf369 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:03.744815 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:03.762956 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1a3e38-bc09-4e63-8766-45cf9e125b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:03.763592 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a3e38-bc09-4e63-8766-45cf9e125b01 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:03.763874 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a3e38-bc09-4e63-8766-45cf9e125b01 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:03.764148 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a3e38-bc09-4e63-8766-45cf9e125b01 None admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:03.764863 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1a3e38-bc09-4e63-8766-45cf9e125b01 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:03.765165 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1a3e38-bc09-4e63-8766-45cf9e125b01 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:03.766577 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1a3e38-bc09-4e63-8766-45cf9e125b01 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:03.766873 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacae074-115a-4e5b-88d6-6e5ff9998838 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:03.787463 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 327/652] 10.222.0.15 () {56 vars in 1133 bytes} [Sun Sep 11 10:34:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:03.787463 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e79ce0b-03d9-4357-bc08-1a80d7ebb0f2 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:03.787463 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e79ce0b-03d9-4357-bc08-1a80d7ebb0f2 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:03.787463 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e79ce0b-03d9-4357-bc08-1a80d7ebb0f2 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:03.787463 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 328/653] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:34:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:34:03.787463 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e79ce0b-03d9-4357-bc08-1a80d7ebb0f2 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:03.805953 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4364738a-985a-4e58-ae01-8a3682aae009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:03.806579 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4364738a-985a-4e58-ae01-8a3682aae009 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:03.806857 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4364738a-985a-4e58-ae01-8a3682aae009 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:03.807127 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4364738a-985a-4e58-ae01-8a3682aae009 None admin] PATH_INFO: `/v3/projects/service` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:03.813860 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4364738a-985a-4e58-ae01-8a3682aae009 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:03.815292 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4364738a-985a-4e58-ae01-8a3682aae009 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:03.826448 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-4364738a-985a-4e58-ae01-8a3682aae009 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:34:03.826448 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:03.826448 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:03.826448 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:03.826448 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:03.826448 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:03.826448 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:03.826448 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:03.826448 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:03.826448 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:03.826448 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:03.826448 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:03.826448 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 10:34:03.826448 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 10:34:03.826448 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 10:34:03.827545 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 10:34:03.827545 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:03.827545 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:03.827545 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:03.827545 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:03.827545 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:03.827545 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:03.827545 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:03.827545 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:03.827545 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:03.827545 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:03.827545 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:03.827545 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:03.827545 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:03.827545 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:03.827545 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:03.827545 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:03.828453 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 10:34:03.828453 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 10:34:03.828453 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 10:34:03.828453 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 10:34:03.828453 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 10:34:03.828453 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 10:34:03.828453 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:34:03.828453 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:03.828453 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 326/654] 10.222.0.15 () {56 vars in 1111 bytes} [Sun Sep 11 10:34:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:03.828453 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4364738a-985a-4e58-ae01-8a3682aae009 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:03.890926 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e79ce0b-03d9-4357-bc08-1a80d7ebb0f2 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:03.904581 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:03.926257 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2844ae26-047c-4497-945c-09cc1861dc67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:03.926762 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4364738a-985a-4e58-ae01-8a3682aae009 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:03.926867 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2844ae26-047c-4497-945c-09cc1861dc67 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:03.927125 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2844ae26-047c-4497-945c-09cc1861dc67 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:03.927396 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2844ae26-047c-4497-945c-09cc1861dc67 None admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:03.928145 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2844ae26-047c-4497-945c-09cc1861dc67 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:03.928447 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2844ae26-047c-4497-945c-09cc1861dc67 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:03.930092 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2844ae26-047c-4497-945c-09cc1861dc67 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:03.947765 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 329/655] 10.222.0.15 () {56 vars in 1139 bytes} [Sun Sep 11 10:34:03 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:03.948567 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2844ae26-047c-4497-945c-09cc1861dc67 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:03.966902 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053a5c87-d8bd-4d9a-92b4-92f8146cc99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:03.967569 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a5c87-d8bd-4d9a-92b4-92f8146cc99e None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:03.967846 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a5c87-d8bd-4d9a-92b4-92f8146cc99e None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:03.968288 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a5c87-d8bd-4d9a-92b4-92f8146cc99e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:03.975855 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053a5c87-d8bd-4d9a-92b4-92f8146cc99e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:03.977539 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053a5c87-d8bd-4d9a-92b4-92f8146cc99e None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:03.980458 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:03.989769 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-053a5c87-d8bd-4d9a-92b4-92f8146cc99e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 11 10:34:03.989769 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:03.989769 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:03.989769 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:03.989769 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:03.989769 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:03.989769 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:03.989769 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:03.989769 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:03.989769 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:03.989769 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:03.989769 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:03.989769 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 10:34:03.989769 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 10:34:03.989769 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 10:34:03.989769 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 10:34:03.990913 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:03.990913 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:03.990913 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 10:34:03.990913 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:34:03.990913 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 10:34:03.990913 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:34:03.990913 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:03.990913 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:03.990913 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:03.990913 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:03.990913 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:03.990913 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:03.990913 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:03.990913 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:03.990913 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:03.990913 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:03.990913 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:03.990913 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:03.992081 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:03.992081 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:03.992081 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 10:34:03.992081 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 10:34:03.992081 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 10:34:03.992081 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 10:34:03.992081 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 10:34:03.992081 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 10:34:03.992081 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 11 10:34:03.992081 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:03.992081 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 327/656] 10.222.0.15 () {56 vars in 1099 bytes} [Sun Sep 11 10:34:03 2022] GET /identity/v3/users/cinder => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:03.992682 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af5d62-2c63-4a5e-83be-21d0fceccd7f None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:03.992682 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af5d62-2c63-4a5e-83be-21d0fceccd7f None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:03.992822 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af5d62-2c63-4a5e-83be-21d0fceccd7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:04.010668 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-73af5d62-2c63-4a5e-83be-21d0fceccd7f None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:34:04.011867 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-73af5d62-2c63-4a5e-83be-21d0fceccd7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:34:04.051829 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2844ae26-047c-4497-945c-09cc1861dc67 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:04.076949 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 328/657] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:34:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:34:04.077770 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73af5d62-2c63-4a5e-83be-21d0fceccd7f None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:04.089040 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb16d6c7-66cb-4534-8ef2-f433d4dacae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:04.089650 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb16d6c7-66cb-4534-8ef2-f433d4dacae0 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:04.089931 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb16d6c7-66cb-4534-8ef2-f433d4dacae0 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:04.090216 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb16d6c7-66cb-4534-8ef2-f433d4dacae0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:04.132753 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:04.165131 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb16d6c7-66cb-4534-8ef2-f433d4dacae0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:04.166510 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb16d6c7-66cb-4534-8ef2-f433d4dacae0 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:04.183066 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73af5d62-2c63-4a5e-83be-21d0fceccd7f None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:04.212644 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:04.221439 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbcdfd4-dceb-4eaf-b9b6-e23ddff9bc16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:04.222047 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbcdfd4-dceb-4eaf-b9b6-e23ddff9bc16 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:04.222371 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbcdfd4-dceb-4eaf-b9b6-e23ddff9bc16 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:04.222649 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbcdfd4-dceb-4eaf-b9b6-e23ddff9bc16 None admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:04.223340 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbcdfd4-dceb-4eaf-b9b6-e23ddff9bc16 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:04.223684 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbcdfd4-dceb-4eaf-b9b6-e23ddff9bc16 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:04.225100 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbcdfd4-dceb-4eaf-b9b6-e23ddff9bc16 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:04.237541 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 329/658] 10.222.0.15 () {56 vars in 1148 bytes} [Sun Sep 11 10:34:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:04.238540 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbcdfd4-dceb-4eaf-b9b6-e23ddff9bc16 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:04.277290 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 330/659] 10.222.0.15 () {58 vars in 1282 bytes} [Sun Sep 11 10:34:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 10:34:04.278522 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb16d6c7-66cb-4534-8ef2-f433d4dacae0 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:04.352595 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:04.356698 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbcdfd4-dceb-4eaf-b9b6-e23ddff9bc16 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:04.384951 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb16d6c7-66cb-4534-8ef2-f433d4dacae0 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:04.390194 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcd15b8-4754-4736-b940-171dea47cf8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:04.390805 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcd15b8-4754-4736-b940-171dea47cf8e None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:04.391086 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcd15b8-4754-4736-b940-171dea47cf8e None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:04.391360 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcd15b8-4754-4736-b940-171dea47cf8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:04.418817 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bc473b-dc7e-4757-8447-640f6e31fe63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:04.419355 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bc473b-dc7e-4757-8447-640f6e31fe63 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:04.419679 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bc473b-dc7e-4757-8447-640f6e31fe63 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:04.419953 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bc473b-dc7e-4757-8447-640f6e31fe63 None admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:04.420671 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bc473b-dc7e-4757-8447-640f6e31fe63 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:04.420973 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bc473b-dc7e-4757-8447-640f6e31fe63 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:04.422385 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bc473b-dc7e-4757-8447-640f6e31fe63 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:04.440596 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 331/660] 10.222.0.15 () {56 vars in 1137 bytes} [Sun Sep 11 10:34:04 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:04.441390 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bc473b-dc7e-4757-8447-640f6e31fe63 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:04.448957 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:04.462215 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcd15b8-4754-4736-b940-171dea47cf8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:04.463712 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcd15b8-4754-4736-b940-171dea47cf8e None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:04.542096 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bc473b-dc7e-4757-8447-640f6e31fe63 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:04.559726 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 330/661] 10.222.0.15 () {58 vars in 1282 bytes} [Sun Sep 11 10:34:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 10:34:04.560542 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcd15b8-4754-4736-b940-171dea47cf8e None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:04.568478 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:04.576638 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fdb199-c75e-407c-baaa-6a547c3b4a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:04.577245 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fdb199-c75e-407c-baaa-6a547c3b4a5e None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:04.577525 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fdb199-c75e-407c-baaa-6a547c3b4a5e None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:04.577793 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fdb199-c75e-407c-baaa-6a547c3b4a5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:04.578511 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fdb199-c75e-407c-baaa-6a547c3b4a5e None admin] RBAC: Adding query filter params (role.id=3cb7ebbc707d453586b3d0f4ab3cf103, scope.project.id=5e1f845f99f04202b83621cb65e0adc2, user.id=1632da5fbe384b5c849be1d4d86c0f1d) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:04.578815 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fdb199-c75e-407c-baaa-6a547c3b4a5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:04.580252 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fdb199-c75e-407c-baaa-6a547c3b4a5e None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:04.586027 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 332/662] 10.222.0.15 () {56 vars in 1374 bytes} [Sun Sep 11 10:34:04 2022] GET /identity/v3/role_assignments?user.id=1632da5fbe384b5c849be1d4d86c0f1d&scope.project.id=5e1f845f99f04202b83621cb65e0adc2&role.id=3cb7ebbc707d453586b3d0f4ab3cf103 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:04.586638 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fdb199-c75e-407c-baaa-6a547c3b4a5e None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:04.664432 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:04.680152 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcd15b8-4754-4736-b940-171dea47cf8e None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:04.704974 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fdb199-c75e-407c-baaa-6a547c3b4a5e None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:04.720357 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bc79c7-ffa4-4219-8dc5-c84ac2cd5e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:04.721067 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc79c7-ffa4-4219-8dc5-c84ac2cd5e24 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:04.721287 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc79c7-ffa4-4219-8dc5-c84ac2cd5e24 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:04.721564 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc79c7-ffa4-4219-8dc5-c84ac2cd5e24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:04.736119 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bc79c7-ffa4-4219-8dc5-c84ac2cd5e24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:04.737582 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bc79c7-ffa4-4219-8dc5-c84ac2cd5e24 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:04.744795 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3b2de3-25e6-4912-8471-22fc83b7e5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:04.745451 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3b2de3-25e6-4912-8471-22fc83b7e5d2 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:04.745734 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3b2de3-25e6-4912-8471-22fc83b7e5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:04.746008 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3b2de3-25e6-4912-8471-22fc83b7e5d2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:04.753952 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-70bc79c7-ffa4-4219-8dc5-c84ac2cd5e24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:04.753952 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:04.753952 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:04.753952 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:04.753952 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:04.753952 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:04.753952 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:04.753952 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:04.753952 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:04.753952 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:04.753952 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:04.753952 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:04.753952 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:34:04.753952 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:34:04.753952 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:34:04.753952 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:34:04.754944 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:04.754944 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:04.754944 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:04.754944 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:04.754944 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:04.754944 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:04.754944 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:04.754944 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:04.754944 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:04.754944 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:04.754944 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:04.754944 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:04.754944 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:04.754944 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:04.754944 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:04.754944 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:04.754944 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:34:04.757555 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:34:04.757555 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:04.757555 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:04.757555 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 331/663] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:34:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:04.757555 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bc79c7-ffa4-4219-8dc5-c84ac2cd5e24 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:04.763603 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3b2de3-25e6-4912-8471-22fc83b7e5d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:34:04.769787 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-dc3b2de3-25e6-4912-8471-22fc83b7e5d2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 10:34:04.769787 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:04.769787 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:04.769787 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:04.769787 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:04.769787 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:04.769787 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:04.769787 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:04.769787 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:04.769787 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:04.769787 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:04.769787 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:04.769787 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:34:04.769787 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:34:04.769787 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:34:04.769787 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:34:04.770961 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:34:04.770961 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:34:04.770961 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:34:04.770961 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:34:04.770961 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:04.770961 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:04.770961 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:04.770961 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:04.770961 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:04.770961 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:04.770961 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:04.770961 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:04.770961 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:04.770961 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:04.770961 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:04.770961 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:04.774024 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:04.774024 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:04.774024 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:04.774024 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:04.774024 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:34:04.774024 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:34:04.774024 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:34:04.774024 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:34:04.774024 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:34:04.774024 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:34:04.774024 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 10:34:04.774024 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:04.774024 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 333/664] 10.222.0.15 () {56 vars in 1102 bytes} [Sun Sep 11 10:34:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:04.774024 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3b2de3-25e6-4912-8471-22fc83b7e5d2 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:04.836474 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:04.874212 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bc79c7-ffa4-4219-8dc5-c84ac2cd5e24 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:04.891929 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3b2de3-25e6-4912-8471-22fc83b7e5d2 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:04.907511 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcf7024-ffd2-4857-92ce-d4736f4dfd34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:04.908117 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcf7024-ffd2-4857-92ce-d4736f4dfd34 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:04.908399 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcf7024-ffd2-4857-92ce-d4736f4dfd34 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:04.908667 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcf7024-ffd2-4857-92ce-d4736f4dfd34 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:04.920171 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:04.931015 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710d9cc8-0529-4808-9c9d-53307181cc7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:04.931679 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d9cc8-0529-4808-9c9d-53307181cc7f None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:04.931965 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d9cc8-0529-4808-9c9d-53307181cc7f None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:04.932241 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d9cc8-0529-4808-9c9d-53307181cc7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:04.946222 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710d9cc8-0529-4808-9c9d-53307181cc7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:04.947715 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710d9cc8-0529-4808-9c9d-53307181cc7f None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:04.965822 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-710d9cc8-0529-4808-9c9d-53307181cc7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:04.965822 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:04.965822 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:04.965822 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:04.965822 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:04.965822 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:04.965822 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:04.965822 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:04.965822 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:04.965822 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:04.965822 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:04.965822 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:04.965822 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:34:04.965822 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:34:04.965822 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:34:04.965822 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:34:04.966974 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:04.966974 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:04.966974 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:04.966974 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:04.966974 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:04.966974 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:04.966974 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:04.966974 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:04.966974 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:04.966974 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:04.966974 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:04.966974 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:04.966974 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:04.966974 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:04.966974 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:04.966974 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:04.966974 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:34:04.968095 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:34:04.968095 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:04.968095 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:04.971573 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 334/665] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:34:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:04.972429 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710d9cc8-0529-4808-9c9d-53307181cc7f None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:04.985938 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcf7024-ffd2-4857-92ce-d4736f4dfd34 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:04.987396 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcf7024-ffd2-4857-92ce-d4736f4dfd34 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:05.081961 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710d9cc8-0529-4808-9c9d-53307181cc7f None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:05.084427 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:05.092918 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 332/666] 10.222.0.15 () {58 vars in 1282 bytes} [Sun Sep 11 10:34:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 10:34:05.094019 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcf7024-ffd2-4857-92ce-d4736f4dfd34 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:05.125279 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf005fb7-2af9-4d50-9207-ab99c29259d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:05.125890 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf005fb7-2af9-4d50-9207-ab99c29259d1 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:05.126167 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf005fb7-2af9-4d50-9207-ab99c29259d1 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:05.126432 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf005fb7-2af9-4d50-9207-ab99c29259d1 None admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:05.127065 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf005fb7-2af9-4d50-9207-ab99c29259d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:05.127363 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf005fb7-2af9-4d50-9207-ab99c29259d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:05.128744 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf005fb7-2af9-4d50-9207-ab99c29259d1 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:05.140559 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 335/667] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:34:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:05.141105 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:05.141358 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf005fb7-2af9-4d50-9207-ab99c29259d1 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:05.207099 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcf7024-ffd2-4857-92ce-d4736f4dfd34 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:05.243765 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e687f2-01ac-4767-b321-ea9a7e58715c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:05.244375 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e687f2-01ac-4767-b321-ea9a7e58715c None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:05.244648 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e687f2-01ac-4767-b321-ea9a7e58715c None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:05.244916 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e687f2-01ac-4767-b321-ea9a7e58715c None admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:05.245562 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e687f2-01ac-4767-b321-ea9a7e58715c None admin] RBAC: Adding query filter params (name=service) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:05.245858 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e687f2-01ac-4767-b321-ea9a7e58715c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:05.247204 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e687f2-01ac-4767-b321-ea9a7e58715c None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:05.248116 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf005fb7-2af9-4d50-9207-ab99c29259d1 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:05.260611 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 333/668] 10.222.0.15 () {56 vars in 1103 bytes} [Sun Sep 11 10:34:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:05.260611 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e687f2-01ac-4767-b321-ea9a7e58715c None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:05.294647 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e177452e-39d7-45a4-aea4-cbf0133d6ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:05.295361 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e177452e-39d7-45a4-aea4-cbf0133d6ac6 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:05.295736 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e177452e-39d7-45a4-aea4-cbf0133d6ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:05.296009 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e177452e-39d7-45a4-aea4-cbf0133d6ac6 None admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:05.296644 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e177452e-39d7-45a4-aea4-cbf0133d6ac6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:05.296948 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e177452e-39d7-45a4-aea4-cbf0133d6ac6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:05.298310 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e177452e-39d7-45a4-aea4-cbf0133d6ac6 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:05.314306 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 336/669] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:34:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:05.320437 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e177452e-39d7-45a4-aea4-cbf0133d6ac6 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:05.345273 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:05.370952 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e687f2-01ac-4767-b321-ea9a7e58715c None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:05.372716 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:05.409583 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84e7361-a0c9-4bb4-8f73-150b5b72bffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:05.410219 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84e7361-a0c9-4bb4-8f73-150b5b72bffb None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:05.410499 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84e7361-a0c9-4bb4-8f73-150b5b72bffb None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:05.410770 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84e7361-a0c9-4bb4-8f73-150b5b72bffb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:05.422498 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84e7361-a0c9-4bb4-8f73-150b5b72bffb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:05.423991 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84e7361-a0c9-4bb4-8f73-150b5b72bffb None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:05.431403 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e177452e-39d7-45a4-aea4-cbf0133d6ac6 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:05.438430 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-d84e7361-a0c9-4bb4-8f73-150b5b72bffb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:05.438430 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:05.438430 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:05.438430 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:05.438430 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:05.438430 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:05.438430 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:05.438430 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:05.438430 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:05.438430 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:05.438430 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:05.438430 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:05.438430 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:34:05.438430 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:34:05.438430 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:34:05.438430 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:34:05.439577 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:05.439577 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:05.439577 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:05.439577 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:05.439577 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:05.439577 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:05.439577 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:05.439577 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:05.439577 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:05.439577 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:05.439577 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:05.439577 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:05.439577 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:05.439577 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:05.439577 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:05.439577 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:05.439577 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:34:05.440495 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:34:05.440495 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:05.440495 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:05.440495 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 334/670] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:34:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:05.440495 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84e7361-a0c9-4bb4-8f73-150b5b72bffb None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:05.474701 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1671dfa8-2d3e-4bc6-9904-0f7849725b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:05.475413 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1671dfa8-2d3e-4bc6-9904-0f7849725b48 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:05.476477 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1671dfa8-2d3e-4bc6-9904-0f7849725b48 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:05.477301 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1671dfa8-2d3e-4bc6-9904-0f7849725b48 None admin] PATH_INFO: `/v3/projects/service` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:05.484277 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1671dfa8-2d3e-4bc6-9904-0f7849725b48 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:05.485803 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1671dfa8-2d3e-4bc6-9904-0f7849725b48 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:05.500117 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-1671dfa8-2d3e-4bc6-9904-0f7849725b48 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:34:05.500117 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:05.500117 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:05.500117 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:05.500117 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:05.500117 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:05.500117 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:05.500117 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:05.500117 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:05.500117 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:05.500117 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:05.500117 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:05.500117 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 10:34:05.500117 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 10:34:05.500117 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 10:34:05.501209 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 10:34:05.501209 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:05.501209 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:05.501209 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:05.501209 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:05.501209 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:05.501209 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:05.501209 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:05.501209 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:05.501209 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:05.501209 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:05.501209 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:05.501209 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:05.501209 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:05.501209 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:05.501209 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:05.501209 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:05.502304 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 10:34:05.502304 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 10:34:05.502304 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 10:34:05.502304 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 10:34:05.502304 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 10:34:05.502304 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 10:34:05.502304 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:34:05.502304 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:05.503307 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 337/671] 10.222.0.15 () {56 vars in 1111 bytes} [Sun Sep 11 10:34:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:05.503978 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1671dfa8-2d3e-4bc6-9904-0f7849725b48 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:05.555636 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84e7361-a0c9-4bb4-8f73-150b5b72bffb None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:05.593408 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:05.595626 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7fa238-c4ae-4bcb-95b7-dba8e21a6031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:05.596360 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7fa238-c4ae-4bcb-95b7-dba8e21a6031 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:05.596707 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7fa238-c4ae-4bcb-95b7-dba8e21a6031 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:05.597046 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7fa238-c4ae-4bcb-95b7-dba8e21a6031 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:05.608953 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7fa238-c4ae-4bcb-95b7-dba8e21a6031 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:05.610444 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7fa238-c4ae-4bcb-95b7-dba8e21a6031 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:05.611431 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1671dfa8-2d3e-4bc6-9904-0f7849725b48 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:05.626005 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-ee7fa238-c4ae-4bcb-95b7-dba8e21a6031 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:05.626005 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:05.626005 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:05.626005 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:05.626005 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:05.626005 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:05.626005 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:05.626005 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:05.626005 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:05.626005 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:05.626005 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:05.626005 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:05.626005 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:34:05.626005 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:34:05.626005 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:34:05.626005 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:34:05.627153 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:05.627153 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:05.627153 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:05.627153 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:05.627153 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:05.627153 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:05.627153 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:05.627153 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:05.627153 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:05.627153 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:05.627153 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:05.627153 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:05.627153 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:05.627153 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:05.627153 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:05.627153 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:05.627153 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:34:05.628105 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:34:05.628105 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:05.628105 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:05.628105 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 335/672] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:34:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:05.628332 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7fa238-c4ae-4bcb-95b7-dba8e21a6031 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:05.638083 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:05.654367 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89acf53-d05a-41c6-bf8d-f8c8dcc36aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:05.655194 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89acf53-d05a-41c6-bf8d-f8c8dcc36aed None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:05.655557 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89acf53-d05a-41c6-bf8d-f8c8dcc36aed None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:05.655935 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89acf53-d05a-41c6-bf8d-f8c8dcc36aed None admin] PATH_INFO: `/v3/projects/service` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:05.662607 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89acf53-d05a-41c6-bf8d-f8c8dcc36aed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:05.664276 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89acf53-d05a-41c6-bf8d-f8c8dcc36aed None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:05.675184 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-c89acf53-d05a-41c6-bf8d-f8c8dcc36aed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:34:05.675184 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:05.675184 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:05.675184 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:05.675184 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:05.675184 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:05.675184 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:05.675184 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:05.675184 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:05.675184 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:05.675184 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:05.675184 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:05.675184 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 10:34:05.675184 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 10:34:05.675184 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 10:34:05.681579 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 10:34:05.681579 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:05.681579 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:05.681579 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:05.681579 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:05.681579 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:05.681579 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:05.681579 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:05.681579 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:05.681579 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:05.681579 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:05.681579 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:05.681579 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:05.681579 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:05.681579 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:05.681579 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:05.681579 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:05.682749 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 10:34:05.682749 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 10:34:05.682749 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 10:34:05.682749 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 10:34:05.682749 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 10:34:05.682749 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 10:34:05.682749 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:34:05.682749 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:05.682749 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 338/673] 10.222.0.15 () {56 vars in 1111 bytes} [Sun Sep 11 10:34:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:05.682749 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89acf53-d05a-41c6-bf8d-f8c8dcc36aed None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:05.743177 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7fa238-c4ae-4bcb-95b7-dba8e21a6031 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:05.780403 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bcb54c-9d29-44c2-bf01-1eea054b1eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:05.781337 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bcb54c-9d29-44c2-bf01-1eea054b1eef None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:05.781623 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bcb54c-9d29-44c2-bf01-1eea054b1eef None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:05.781893 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bcb54c-9d29-44c2-bf01-1eea054b1eef None admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:05.782529 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bcb54c-9d29-44c2-bf01-1eea054b1eef None admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:05.782831 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bcb54c-9d29-44c2-bf01-1eea054b1eef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:05.784279 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bcb54c-9d29-44c2-bf01-1eea054b1eef None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:05.789421 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89acf53-d05a-41c6-bf8d-f8c8dcc36aed None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:05.796365 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 336/674] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:34:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:05.797178 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bcb54c-9d29-44c2-bf01-1eea054b1eef None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:05.830200 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5900ba-51ba-4849-a1ce-782c37b9d89c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:05.830702 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5900ba-51ba-4849-a1ce-782c37b9d89c None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:05.831006 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5900ba-51ba-4849-a1ce-782c37b9d89c None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:05.831253 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5900ba-51ba-4849-a1ce-782c37b9d89c None admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:05.831973 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5900ba-51ba-4849-a1ce-782c37b9d89c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:05.832280 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5900ba-51ba-4849-a1ce-782c37b9d89c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:05.833693 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5900ba-51ba-4849-a1ce-782c37b9d89c None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:05.851408 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 339/675] 10.222.0.15 () {56 vars in 1148 bytes} [Sun Sep 11 10:34:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:05.851408 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5900ba-51ba-4849-a1ce-782c37b9d89c None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:05.854404 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:05.873583 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:05.910727 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bcb54c-9d29-44c2-bf01-1eea054b1eef None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:05.945400 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b249ca-bd77-472c-9c75-6ab04fda8272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:05.946023 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b249ca-bd77-472c-9c75-6ab04fda8272 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:05.946303 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b249ca-bd77-472c-9c75-6ab04fda8272 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:05.946575 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b249ca-bd77-472c-9c75-6ab04fda8272 None admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:05.947209 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b249ca-bd77-472c-9c75-6ab04fda8272 None admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:05.947541 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b249ca-bd77-472c-9c75-6ab04fda8272 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:05.948910 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b249ca-bd77-472c-9c75-6ab04fda8272 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:05.957904 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5900ba-51ba-4849-a1ce-782c37b9d89c None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:05.965917 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 337/676] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:34:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:05.965917 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b249ca-bd77-472c-9c75-6ab04fda8272 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:05.994795 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e220be90-801a-4d52-8b11-87e3bce41df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:05.995463 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e220be90-801a-4d52-8b11-87e3bce41df4 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:05.995814 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e220be90-801a-4d52-8b11-87e3bce41df4 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:05.996164 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e220be90-801a-4d52-8b11-87e3bce41df4 None admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:05.996829 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e220be90-801a-4d52-8b11-87e3bce41df4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:05.997130 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e220be90-801a-4d52-8b11-87e3bce41df4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:05.998529 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e220be90-801a-4d52-8b11-87e3bce41df4 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:06.015829 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 340/677] 10.222.0.15 () {56 vars in 1148 bytes} [Sun Sep 11 10:34:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:06.015829 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e220be90-801a-4d52-8b11-87e3bce41df4 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:06.070116 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b249ca-bd77-472c-9c75-6ab04fda8272 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:06.081989 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:06.096722 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:06.107198 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a0ef2b-33e3-489b-92ef-f0676880ffcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:06.107977 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a0ef2b-33e3-489b-92ef-f0676880ffcd None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:06.108362 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a0ef2b-33e3-489b-92ef-f0676880ffcd None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:06.108766 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a0ef2b-33e3-489b-92ef-f0676880ffcd None admin] PATH_INFO: `/v3/projects/service` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:06.115323 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a0ef2b-33e3-489b-92ef-f0676880ffcd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:06.117596 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a0ef2b-33e3-489b-92ef-f0676880ffcd None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:06.123954 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e220be90-801a-4d52-8b11-87e3bce41df4 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:06.128718 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-d4a0ef2b-33e3-489b-92ef-f0676880ffcd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:34:06.128718 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:06.128718 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:06.128718 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:06.128718 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:06.128718 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:06.128718 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:06.128718 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:06.128718 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:06.128718 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:06.128718 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:06.128718 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:06.128718 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 10:34:06.128718 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 10:34:06.128718 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 10:34:06.129634 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 10:34:06.129634 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:06.129634 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:06.129634 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:06.129634 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:06.129634 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:06.129634 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:06.129634 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:06.129634 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:06.129634 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:06.129634 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:06.129634 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:06.129634 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:06.129634 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:06.129634 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:06.129634 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:06.129634 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:06.130536 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 10:34:06.130536 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 10:34:06.130536 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 10:34:06.130536 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 10:34:06.130536 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 10:34:06.130536 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 10:34:06.130536 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:34:06.130536 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:06.135276 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 338/678] 10.222.0.15 () {56 vars in 1111 bytes} [Sun Sep 11 10:34:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:06.136044 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a0ef2b-33e3-489b-92ef-f0676880ffcd None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:06.160344 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8d6616-dbee-46c4-8a18-e2a6c3fb4ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:06.161265 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d6616-dbee-46c4-8a18-e2a6c3fb4ff5 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:06.161547 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d6616-dbee-46c4-8a18-e2a6c3fb4ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:06.161811 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d6616-dbee-46c4-8a18-e2a6c3fb4ff5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:06.162525 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8d6616-dbee-46c4-8a18-e2a6c3fb4ff5 None admin] RBAC: Adding query filter params (role.id=3cb7ebbc707d453586b3d0f4ab3cf103, scope.project.id=5e1f845f99f04202b83621cb65e0adc2, user.id=2bd5356829b6454993bab9bad2f79fed) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:06.162831 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8d6616-dbee-46c4-8a18-e2a6c3fb4ff5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:06.165059 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8d6616-dbee-46c4-8a18-e2a6c3fb4ff5 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:06.170766 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 341/679] 10.222.0.15 () {56 vars in 1374 bytes} [Sun Sep 11 10:34:06 2022] GET /identity/v3/role_assignments?user.id=2bd5356829b6454993bab9bad2f79fed&scope.project.id=5e1f845f99f04202b83621cb65e0adc2&role.id=3cb7ebbc707d453586b3d0f4ab3cf103 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:06.171556 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8d6616-dbee-46c4-8a18-e2a6c3fb4ff5 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:06.255992 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a0ef2b-33e3-489b-92ef-f0676880ffcd None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:06.293629 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b259acd-7a9b-4d0a-a0a8-d5ca1518e9db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:06.294232 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b259acd-7a9b-4d0a-a0a8-d5ca1518e9db None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:06.294511 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b259acd-7a9b-4d0a-a0a8-d5ca1518e9db None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:06.294783 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b259acd-7a9b-4d0a-a0a8-d5ca1518e9db None admin] PATH_INFO: `/v3/users/swift` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:06.302094 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8d6616-dbee-46c4-8a18-e2a6c3fb4ff5 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:06.303176 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b259acd-7a9b-4d0a-a0a8-d5ca1518e9db None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:06.305530 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b259acd-7a9b-4d0a-a0a8-d5ca1518e9db None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:06.317787 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-4b259acd-7a9b-4d0a-a0a8-d5ca1518e9db None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 11 10:34:06.317787 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:06.317787 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:06.317787 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:06.317787 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:06.317787 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:06.317787 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:06.317787 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:06.317787 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:06.317787 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:06.317787 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:06.317787 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:06.317787 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 10:34:06.317787 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 10:34:06.317787 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 10:34:06.317787 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 10:34:06.318713 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:06.318713 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:06.318713 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 10:34:06.318713 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:34:06.318713 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 10:34:06.318713 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:34:06.318713 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:06.318713 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:06.318713 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:06.318713 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:06.318713 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:06.318713 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:06.318713 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:06.318713 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:06.318713 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:06.318713 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:06.318713 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:06.318713 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:06.320478 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:06.320478 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:06.320478 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 10:34:06.320478 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 10:34:06.320478 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 10:34:06.320478 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 10:34:06.320478 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 10:34:06.320478 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 10:34:06.320478 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 11 10:34:06.320478 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:06.324014 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 339/680] 10.222.0.15 () {56 vars in 1096 bytes} [Sun Sep 11 10:34:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:06.324949 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b259acd-7a9b-4d0a-a0a8-d5ca1518e9db None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:06.341988 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:06.353702 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d43243a-9e21-46f7-9dba-298a0cd73edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:06.354733 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d43243a-9e21-46f7-9dba-298a0cd73edc None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:06.355312 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d43243a-9e21-46f7-9dba-298a0cd73edc None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:06.355816 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d43243a-9e21-46f7-9dba-298a0cd73edc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:06.357411 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d43243a-9e21-46f7-9dba-298a0cd73edc None admin] RBAC: Adding query filter params (role.id=3cb7ebbc707d453586b3d0f4ab3cf103, scope.project.id=5e1f845f99f04202b83621cb65e0adc2, user.id=40072bc8dcb5454382a93e20b63f8594) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:06.357877 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d43243a-9e21-46f7-9dba-298a0cd73edc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:06.359689 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d43243a-9e21-46f7-9dba-298a0cd73edc None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:06.367480 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 342/681] 10.222.0.15 () {56 vars in 1374 bytes} [Sun Sep 11 10:34:06 2022] GET /identity/v3/role_assignments?user.id=40072bc8dcb5454382a93e20b63f8594&scope.project.id=5e1f845f99f04202b83621cb65e0adc2&role.id=3cb7ebbc707d453586b3d0f4ab3cf103 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:06.368527 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d43243a-9e21-46f7-9dba-298a0cd73edc None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:06.386534 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:06.462857 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b259acd-7a9b-4d0a-a0a8-d5ca1518e9db None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:06.518843 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d43243a-9e21-46f7-9dba-298a0cd73edc None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:06.546677 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3c0c50-39bc-4c3d-954c-8f405ac9ddcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:06.547410 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3c0c50-39bc-4c3d-954c-8f405ac9ddcb None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:06.547725 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3c0c50-39bc-4c3d-954c-8f405ac9ddcb None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:06.548002 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3c0c50-39bc-4c3d-954c-8f405ac9ddcb None admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:06.548828 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3c0c50-39bc-4c3d-954c-8f405ac9ddcb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:06.549143 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3c0c50-39bc-4c3d-954c-8f405ac9ddcb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:06.550653 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3c0c50-39bc-4c3d-954c-8f405ac9ddcb None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:06.557365 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a1177f-ab80-4857-9016-3f85cb16703c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:06.558079 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a1177f-ab80-4857-9016-3f85cb16703c None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:06.558430 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a1177f-ab80-4857-9016-3f85cb16703c None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:06.558772 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a1177f-ab80-4857-9016-3f85cb16703c None admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:06.559617 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a1177f-ab80-4857-9016-3f85cb16703c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:06.560007 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a1177f-ab80-4857-9016-3f85cb16703c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:06.561539 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a1177f-ab80-4857-9016-3f85cb16703c None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:06.567576 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 340/682] 10.222.0.15 () {56 vars in 1148 bytes} [Sun Sep 11 10:34:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:06.572347 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3c0c50-39bc-4c3d-954c-8f405ac9ddcb None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:06.582760 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 343/683] 10.222.0.15 () {56 vars in 1135 bytes} [Sun Sep 11 10:34:06 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:06.588993 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a1177f-ab80-4857-9016-3f85cb16703c None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:06.643901 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:06.699990 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3c0c50-39bc-4c3d-954c-8f405ac9ddcb None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:06.707956 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:06.712537 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a1177f-ab80-4857-9016-3f85cb16703c None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:06.740916 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b09650-a3a5-4c85-9c36-44fc4bad1dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:06.741675 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b09650-a3a5-4c85-9c36-44fc4bad1dc9 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:06.742124 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b09650-a3a5-4c85-9c36-44fc4bad1dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:06.742626 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b09650-a3a5-4c85-9c36-44fc4bad1dc9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:06.743310 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b09650-a3a5-4c85-9c36-44fc4bad1dc9 None admin] RBAC: Adding query filter params (role.id=3cb7ebbc707d453586b3d0f4ab3cf103, scope.project.id=5e1f845f99f04202b83621cb65e0adc2, user.id=0bbb481abb8e43788e3412ac7d908ad3) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:06.743901 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b09650-a3a5-4c85-9c36-44fc4bad1dc9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:06.746383 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b09650-a3a5-4c85-9c36-44fc4bad1dc9 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:06.755518 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 341/684] 10.222.0.15 () {56 vars in 1374 bytes} [Sun Sep 11 10:34:06 2022] GET /identity/v3/role_assignments?user.id=0bbb481abb8e43788e3412ac7d908ad3&scope.project.id=5e1f845f99f04202b83621cb65e0adc2&role.id=3cb7ebbc707d453586b3d0f4ab3cf103 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:06.760127 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b377a3-7a83-44bb-ab2b-444da5d89500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:06.760767 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b377a3-7a83-44bb-ab2b-444da5d89500 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:06.761109 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b377a3-7a83-44bb-ab2b-444da5d89500 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:06.761456 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b377a3-7a83-44bb-ab2b-444da5d89500 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:06.851161 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b377a3-7a83-44bb-ab2b-444da5d89500 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:06.852805 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b377a3-7a83-44bb-ab2b-444da5d89500 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:06.889302 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf25b3f-43ff-43d5-9f4e-c64f48cf3026 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:06.889606 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf25b3f-43ff-43d5-9f4e-c64f48cf3026 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:06.890018 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf25b3f-43ff-43d5-9f4e-c64f48cf3026 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:06.895710 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 342/685] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:34:06 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 10:34:06.897036 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a792b7-0894-4500-85ac-d6cfc6bcf457 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:06.897469 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a792b7-0894-4500-85ac-d6cfc6bcf457 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:06.897815 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a792b7-0894-4500-85ac-d6cfc6bcf457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:06.916893 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-f1a792b7-0894-4500-85ac-d6cfc6bcf457 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:34:06.918096 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-f1a792b7-0894-4500-85ac-d6cfc6bcf457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:34:06.929475 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:07.030341 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 344/686] 10.222.0.15 () {58 vars in 1282 bytes} [Sun Sep 11 10:34:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 11 10:34:07.036991 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 343/687] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:34:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 10:34:07.039048 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b377a3-7a83-44bb-ab2b-444da5d89500 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:07.157938 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b377a3-7a83-44bb-ab2b-444da5d89500 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:07.200284 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b174e4-5054-4307-b665-ce5a0fa4cd92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:07.201409 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b174e4-5054-4307-b665-ce5a0fa4cd92 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:07.201989 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b174e4-5054-4307-b665-ce5a0fa4cd92 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:07.202563 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b174e4-5054-4307-b665-ce5a0fa4cd92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:07.215596 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b174e4-5054-4307-b665-ce5a0fa4cd92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:07.217267 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b174e4-5054-4307-b665-ce5a0fa4cd92 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:07.233004 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-55b174e4-5054-4307-b665-ce5a0fa4cd92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:07.233004 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:07.233004 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:07.233004 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:07.233004 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:07.233004 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:07.233004 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:07.233004 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:07.233004 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:07.233004 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:07.233004 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:07.233004 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:07.233004 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:34:07.233004 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:34:07.233004 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:34:07.233004 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:34:07.234215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:07.234215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:07.234215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:07.234215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:07.234215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:07.234215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:07.234215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:07.234215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:07.234215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:07.234215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:07.234215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:07.234215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:07.234215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:07.234215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:07.234215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:07.234215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:07.234215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:34:07.235338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:34:07.235338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:07.235338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:07.237044 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 345/688] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:34:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 10:34:07.242996 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a792b7-0894-4500-85ac-d6cfc6bcf457 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:07.347791 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:07.352436 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a792b7-0894-4500-85ac-d6cfc6bcf457 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:07.359626 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b174e4-5054-4307-b665-ce5a0fa4cd92 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:07.365132 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:07.392786 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b8d92e-9561-4eb7-a7f3-5704fe7b89af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:07.393416 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b8d92e-9561-4eb7-a7f3-5704fe7b89af None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:07.393697 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b8d92e-9561-4eb7-a7f3-5704fe7b89af None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:07.393971 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b8d92e-9561-4eb7-a7f3-5704fe7b89af None admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:07.394970 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b8d92e-9561-4eb7-a7f3-5704fe7b89af None admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:07.395275 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b8d92e-9561-4eb7-a7f3-5704fe7b89af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:07.397862 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b8d92e-9561-4eb7-a7f3-5704fe7b89af None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:07.421805 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 344/689] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:34:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:07.429306 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b8d92e-9561-4eb7-a7f3-5704fe7b89af None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:07.560632 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b174e4-5054-4307-b665-ce5a0fa4cd92 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:07.602532 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b8d92e-9561-4eb7-a7f3-5704fe7b89af None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:07.629517 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c59e3fb-1dc7-4c52-85b5-3535631588eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:07.630201 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59e3fb-1dc7-4c52-85b5-3535631588eb None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:07.630493 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59e3fb-1dc7-4c52-85b5-3535631588eb None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:07.630765 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59e3fb-1dc7-4c52-85b5-3535631588eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:07.648923 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0f3c9e-2719-4e3c-b837-ea4e2f1a871a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:07.649872 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0f3c9e-2719-4e3c-b837-ea4e2f1a871a None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:07.650377 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0f3c9e-2719-4e3c-b837-ea4e2f1a871a None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:07.650854 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0f3c9e-2719-4e3c-b837-ea4e2f1a871a None admin] PATH_INFO: `/v3/projects/service` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:07.658846 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0f3c9e-2719-4e3c-b837-ea4e2f1a871a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:07.659002 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c59e3fb-1dc7-4c52-85b5-3535631588eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:07.660461 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c59e3fb-1dc7-4c52-85b5-3535631588eb None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:07.661083 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0f3c9e-2719-4e3c-b837-ea4e2f1a871a None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:07.681096 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-1b0f3c9e-2719-4e3c-b837-ea4e2f1a871a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:34:07.681096 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:07.681096 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:07.681096 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:07.681096 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:07.681096 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:07.681096 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:07.681096 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:07.681096 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:07.681096 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:07.681096 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:07.681096 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:07.681096 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 10:34:07.681096 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 10:34:07.681096 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 10:34:07.682017 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 10:34:07.682017 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:07.682017 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:07.682017 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:07.682017 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:07.682017 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:07.682017 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:07.682017 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:07.682017 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:07.682017 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:07.682017 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:07.682017 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:07.682017 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:07.682017 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:07.682017 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:07.682017 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:07.682017 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:07.682958 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 10:34:07.682958 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 10:34:07.682958 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 10:34:07.682958 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 10:34:07.682958 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 10:34:07.682958 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 10:34:07.682958 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:34:07.682958 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:07.691282 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-0c59e3fb-1dc7-4c52-85b5-3535631588eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:07.691282 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:07.691282 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:07.691282 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:07.691282 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:07.691282 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:07.691282 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:07.691282 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:07.691282 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:07.691282 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:07.691282 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:07.691282 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:07.691282 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:34:07.691282 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:34:07.691282 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:34:07.691282 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:34:07.692253 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:07.692253 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:07.692253 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:07.692253 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:07.692253 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:07.692253 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:07.692253 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:07.692253 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:07.692253 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:07.692253 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:07.692253 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:07.692253 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:07.692253 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:07.692253 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:07.692253 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:07.692253 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:07.692253 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:34:07.693238 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:34:07.693238 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:07.693238 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:07.696025 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 345/690] 10.222.0.15 () {56 vars in 1111 bytes} [Sun Sep 11 10:34:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:07.697745 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 346/691] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:34:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:07.705951 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0f3c9e-2719-4e3c-b837-ea4e2f1a871a None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:07.707186 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c59e3fb-1dc7-4c52-85b5-3535631588eb None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:07.748625 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:07.773620 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:07.820951 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c59e3fb-1dc7-4c52-85b5-3535631588eb None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:07.856983 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c579224f-c314-4bfa-9111-90afcf8ea175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:07.857694 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c579224f-c314-4bfa-9111-90afcf8ea175 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:07.858050 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c579224f-c314-4bfa-9111-90afcf8ea175 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:07.858416 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c579224f-c314-4bfa-9111-90afcf8ea175 None admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:07.859177 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c579224f-c314-4bfa-9111-90afcf8ea175 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:07.859603 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c579224f-c314-4bfa-9111-90afcf8ea175 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:07.861110 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c579224f-c314-4bfa-9111-90afcf8ea175 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:07.872022 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0f3c9e-2719-4e3c-b837-ea4e2f1a871a None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:07.874455 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 347/692] 10.222.0.15 () {56 vars in 1148 bytes} [Sun Sep 11 10:34:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:07.880237 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c579224f-c314-4bfa-9111-90afcf8ea175 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:07.949798 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0187d2b4-17a5-416b-b8bd-aab7a08a20cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:07.950426 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0187d2b4-17a5-416b-b8bd-aab7a08a20cd None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:07.950713 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0187d2b4-17a5-416b-b8bd-aab7a08a20cd None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:07.950986 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0187d2b4-17a5-416b-b8bd-aab7a08a20cd None admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:07.951660 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0187d2b4-17a5-416b-b8bd-aab7a08a20cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:07.952021 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0187d2b4-17a5-416b-b8bd-aab7a08a20cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:07.953388 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0187d2b4-17a5-416b-b8bd-aab7a08a20cd None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:07.972798 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 346/693] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:34:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:34:07.981818 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0187d2b4-17a5-416b-b8bd-aab7a08a20cd None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:08.022964 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c579224f-c314-4bfa-9111-90afcf8ea175 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:08.059840 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:08.075336 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc48f62-311c-4bde-a705-2982b2b30dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:08.076401 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc48f62-311c-4bde-a705-2982b2b30dac None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:08.076882 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc48f62-311c-4bde-a705-2982b2b30dac None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:08.077349 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc48f62-311c-4bde-a705-2982b2b30dac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:08.078325 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc48f62-311c-4bde-a705-2982b2b30dac None admin] RBAC: Adding query filter params (role.id=3cb7ebbc707d453586b3d0f4ab3cf103, scope.project.id=5e1f845f99f04202b83621cb65e0adc2, user.id=cc7ca566e66a45b29e3d02f47bf84981) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:08.078866 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc48f62-311c-4bde-a705-2982b2b30dac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:08.080552 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc48f62-311c-4bde-a705-2982b2b30dac None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:08.095336 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 348/694] 10.222.0.15 () {56 vars in 1374 bytes} [Sun Sep 11 10:34:07 2022] GET /identity/v3/role_assignments?user.id=cc7ca566e66a45b29e3d02f47bf84981&scope.project.id=5e1f845f99f04202b83621cb65e0adc2&role.id=3cb7ebbc707d453586b3d0f4ab3cf103 => generated 608 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:08.151811 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:08.165315 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0187d2b4-17a5-416b-b8bd-aab7a08a20cd None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:08.224388 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563d91f9-9c3d-460d-ad55-89d50f53514a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:08.225932 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563d91f9-9c3d-460d-ad55-89d50f53514a None admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:08.226419 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563d91f9-9c3d-460d-ad55-89d50f53514a None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:08.226899 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563d91f9-9c3d-460d-ad55-89d50f53514a None admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:08.228043 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563d91f9-9c3d-460d-ad55-89d50f53514a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:08.229765 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563d91f9-9c3d-460d-ad55-89d50f53514a None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:08.258770 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 347/695] 10.222.0.15 () {60 vars in 1132 bytes} [Sun Sep 11 10:34:07 2022] POST /identity/v3/users => generated 254 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 10:34:09.000199 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891c236e-9d94-4ec0-8458-1a8fd314c6da None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:09.000720 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891c236e-9d94-4ec0-8458-1a8fd314c6da None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:09.000842 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891c236e-9d94-4ec0-8458-1a8fd314c6da None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:09.002356 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 349/696] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:34:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:34:09.007794 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c2bad5-c1ef-4371-970e-1562cfe63e42 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:09.008237 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c2bad5-c1ef-4371-970e-1562cfe63e42 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:09.008417 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c2bad5-c1ef-4371-970e-1562cfe63e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:09.016442 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a90ae5-735d-4714-b9fd-a48b480e92e1 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:09.016570 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a90ae5-735d-4714-b9fd-a48b480e92e1 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:09.016949 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a90ae5-735d-4714-b9fd-a48b480e92e1 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:09.018752 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 350/697] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:34:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 10:34:09.024969 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81536745-1a52-45ce-b738-25d353cb8d11 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:09.025257 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81536745-1a52-45ce-b738-25d353cb8d11 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:09.025535 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81536745-1a52-45ce-b738-25d353cb8d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:09.030899 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-78c2bad5-c1ef-4371-970e-1562cfe63e42 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:34:09.032127 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-78c2bad5-c1ef-4371-970e-1562cfe63e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:34:09.050570 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-81536745-1a52-45ce-b738-25d353cb8d11 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:34:09.051794 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-81536745-1a52-45ce-b738-25d353cb8d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:34:09.177785 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 348/698] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:34:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:34:09.206697 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 351/699] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:34:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 10:34:09.538594 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c2bad5-c1ef-4371-970e-1562cfe63e42 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:09.639578 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:09.703710 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81536745-1a52-45ce-b738-25d353cb8d11 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:09.708557 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c2bad5-c1ef-4371-970e-1562cfe63e42 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:09.799916 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cef2d5b-2f71-4540-9b2e-88fde57a7d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:09.800583 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cef2d5b-2f71-4540-9b2e-88fde57a7d38 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:09.800865 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cef2d5b-2f71-4540-9b2e-88fde57a7d38 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:09.801139 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cef2d5b-2f71-4540-9b2e-88fde57a7d38 None admin] PATH_INFO: `/v3/services/network` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:09.801741 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cef2d5b-2f71-4540-9b2e-88fde57a7d38 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:09.804265 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cef2d5b-2f71-4540-9b2e-88fde57a7d38 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:09.816480 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-5cef2d5b-2f71-4540-9b2e-88fde57a7d38 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 11 10:34:09.816480 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:09.816480 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 10:34:09.816480 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 10:34:09.816480 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 10:34:09.816480 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 10:34:09.816480 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 10:34:09.816480 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 10:34:09.816480 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 11 10:34:09.816480 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:09.816480 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 10:34:09.816480 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:09.816480 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:09.816480 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:09.816480 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:09.816480 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:09.816480 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:09.817595 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:09.817595 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:09.817595 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:09.817595 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:09.817595 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:09.817595 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:09.817595 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 10:34:09.817595 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 10:34:09.817595 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 10:34:09.817595 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 10:34:09.817595 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:09.817595 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:09.817595 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:09.817595 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:09.817595 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:09.817595 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:09.818481 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:09.818481 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:09.818481 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:09.818481 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:09.818481 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:09.818481 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:09.818481 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:09.818481 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:09.818481 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:09.818481 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:09.818481 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 10:34:09.818481 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 10:34:09.818481 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 11 10:34:09.818481 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:09.822363 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 349/700] 10.222.0.15 () {56 vars in 1111 bytes} [Sun Sep 11 10:34:09 2022] GET /identity/v3/services/network => generated 88 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:09.823975 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbcccc-be92-446e-94f0-56fba8ca4a1d None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:09.825748 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbcccc-be92-446e-94f0-56fba8ca4a1d None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:09.826264 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbcccc-be92-446e-94f0-56fba8ca4a1d None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:09.832188 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 350/701] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:34:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:34:09.832575 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fbcccc-be92-446e-94f0-56fba8ca4a1d None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:09.898751 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81536745-1a52-45ce-b738-25d353cb8d11 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:09.911027 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:09.947451 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809980ee-01ee-440b-897e-dd85e3aee4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:09.947451 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809980ee-01ee-440b-897e-dd85e3aee4be None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:09.947451 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809980ee-01ee-440b-897e-dd85e3aee4be None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:09.947451 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809980ee-01ee-440b-897e-dd85e3aee4be None admin] PATH_INFO: `/v3/roles/admin` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:09.961146 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809980ee-01ee-440b-897e-dd85e3aee4be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:34:09.967391 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-809980ee-01ee-440b-897e-dd85e3aee4be None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 10:34:09.967391 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:09.967391 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:09.967391 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:09.967391 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:09.967391 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:09.967391 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:09.967391 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:09.967391 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:09.967391 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:09.967391 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:09.967391 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:09.967391 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:34:09.967391 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:34:09.967391 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:34:09.967391 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:34:09.968385 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:34:09.968385 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:34:09.968385 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:34:09.968385 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:34:09.968385 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:09.968385 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:09.968385 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:09.968385 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:09.968385 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:09.968385 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:09.968385 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:09.968385 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:09.968385 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:09.968385 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:09.968385 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:09.968385 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:09.969653 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:09.969653 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:09.969653 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:09.969653 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:09.969653 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:34:09.969653 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:34:09.969653 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:34:09.969653 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:34:09.969653 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:34:09.969653 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:34:09.969653 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 10:34:09.969653 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:09.969653 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 352/702] 10.222.0.15 () {56 vars in 1096 bytes} [Sun Sep 11 10:34:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 10:34:09.970716 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d613fa8d-aacc-4709-b0ca-33033bdb41ed None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:09.970716 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d613fa8d-aacc-4709-b0ca-33033bdb41ed None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:09.970716 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d613fa8d-aacc-4709-b0ca-33033bdb41ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:10.003668 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-d613fa8d-aacc-4709-b0ca-33033bdb41ed None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:34:10.007454 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-d613fa8d-aacc-4709-b0ca-33033bdb41ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:34:10.044764 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fbcccc-be92-446e-94f0-56fba8ca4a1d None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:10.108090 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:10.114607 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e986054-c9a7-42a6-acf3-bc66d4458b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:10.115223 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e986054-c9a7-42a6-acf3-bc66d4458b1e None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:10.115529 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e986054-c9a7-42a6-acf3-bc66d4458b1e None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:10.115802 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e986054-c9a7-42a6-acf3-bc66d4458b1e None admin] PATH_INFO: `/v3/services` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:10.119445 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e986054-c9a7-42a6-acf3-bc66d4458b1e None admin] RBAC: Adding query filter params (name=network) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:10.119445 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e986054-c9a7-42a6-acf3-bc66d4458b1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:10.119445 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e986054-c9a7-42a6-acf3-bc66d4458b1e None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:10.132316 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 351/703] 10.222.0.15 () {56 vars in 1112 bytes} [Sun Sep 11 10:34:09 2022] GET /identity/v3/services?name=network => generated 124 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:10.133245 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e986054-c9a7-42a6-acf3-bc66d4458b1e None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:10.146971 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 353/704] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:34:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:34:10.153365 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d613fa8d-aacc-4709-b0ca-33033bdb41ed None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:10.395457 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:10.407088 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e986054-c9a7-42a6-acf3-bc66d4458b1e None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:10.411225 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d613fa8d-aacc-4709-b0ca-33033bdb41ed None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:10.485292 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e218b10-42c7-4683-a1d7-8876b540cb7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:10.486287 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e218b10-42c7-4683-a1d7-8876b540cb7c None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:10.486772 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e218b10-42c7-4683-a1d7-8876b540cb7c None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:10.487247 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e218b10-42c7-4683-a1d7-8876b540cb7c None admin] PATH_INFO: `/v3/services` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:10.488157 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e218b10-42c7-4683-a1d7-8876b540cb7c None admin] RBAC: Adding query filter params (type=network) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:10.488690 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e218b10-42c7-4683-a1d7-8876b540cb7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:10.490529 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e218b10-42c7-4683-a1d7-8876b540cb7c None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:10.501690 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 354/705] 10.222.0.15 () {56 vars in 1112 bytes} [Sun Sep 11 10:34:10 2022] GET /identity/v3/services?type=network => generated 124 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:10.520928 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec40e15c-5af6-4b89-9f3e-2e33bc9487e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:10.521572 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec40e15c-5af6-4b89-9f3e-2e33bc9487e6 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:10.521852 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec40e15c-5af6-4b89-9f3e-2e33bc9487e6 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:10.522122 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec40e15c-5af6-4b89-9f3e-2e33bc9487e6 None admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:10.523022 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec40e15c-5af6-4b89-9f3e-2e33bc9487e6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:10.525969 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec40e15c-5af6-4b89-9f3e-2e33bc9487e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:10.525969 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec40e15c-5af6-4b89-9f3e-2e33bc9487e6 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:10.543851 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 352/706] 10.222.0.15 () {56 vars in 1099 bytes} [Sun Sep 11 10:34:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 411 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:10.552971 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e218b10-42c7-4683-a1d7-8876b540cb7c None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:10.593351 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec40e15c-5af6-4b89-9f3e-2e33bc9487e6 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:10.747164 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:10.802773 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e218b10-42c7-4683-a1d7-8876b540cb7c None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:10.836262 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec40e15c-5af6-4b89-9f3e-2e33bc9487e6 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:10.889563 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670933cb-943a-4214-9ea6-6bf796a67aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:10.890231 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670933cb-943a-4214-9ea6-6bf796a67aeb None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:10.890513 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670933cb-943a-4214-9ea6-6bf796a67aeb None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:10.890786 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670933cb-943a-4214-9ea6-6bf796a67aeb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:10.895456 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a109037-a55e-4664-8b19-69f0012d09ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:10.895456 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a109037-a55e-4664-8b19-69f0012d09ea None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:10.895456 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a109037-a55e-4664-8b19-69f0012d09ea None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:10.895456 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a109037-a55e-4664-8b19-69f0012d09ea None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:10.895834 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a109037-a55e-4664-8b19-69f0012d09ea None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:10.895933 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a109037-a55e-4664-8b19-69f0012d09ea None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:10.916908 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-7a109037-a55e-4664-8b19-69f0012d09ea None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 11 10:34:10.916908 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:10.916908 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 10:34:10.916908 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 10:34:10.916908 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 10:34:10.916908 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 10:34:10.916908 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 10:34:10.916908 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 10:34:10.916908 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 11 10:34:10.916908 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:10.916908 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 10:34:10.916908 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:10.916908 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:10.916908 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:10.916908 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:10.916908 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:10.916908 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:10.918447 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:10.918447 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:10.918447 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:10.918447 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:10.918447 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:10.918447 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:10.918447 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 10:34:10.918447 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 10:34:10.918447 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 10:34:10.918447 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 10:34:10.918447 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:10.918447 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:10.918447 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:10.918447 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:10.918447 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:10.918447 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:10.919763 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:10.919763 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:10.919763 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:10.919763 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:10.919763 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:10.919763 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:10.919763 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:10.919763 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:10.919763 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:10.919763 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:10.919763 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 10:34:10.919763 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 10:34:10.919763 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 11 10:34:10.919763 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:10.919763 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 353/707] 10.222.0.15 () {56 vars in 1129 bytes} [Sun Sep 11 10:34:10 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:10.924388 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670933cb-943a-4214-9ea6-6bf796a67aeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:10.924388 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670933cb-943a-4214-9ea6-6bf796a67aeb None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:10.926348 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a109037-a55e-4664-8b19-69f0012d09ea None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:10.961807 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-670933cb-943a-4214-9ea6-6bf796a67aeb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:10.961807 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:10.961807 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:10.961807 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:10.961807 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:10.961807 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:10.961807 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:10.961807 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:10.961807 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:10.961807 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:10.961807 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:10.961807 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:10.961807 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:34:10.961807 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:34:10.961807 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:34:10.961807 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:34:10.962829 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:10.962829 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:10.962829 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:10.962829 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:10.962829 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:10.962829 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:10.962829 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:10.962829 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:10.962829 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:10.962829 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:10.962829 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:10.962829 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:10.962829 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:10.962829 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:10.962829 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:10.962829 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:10.962829 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:34:10.963772 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:34:10.963772 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:10.963772 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:10.967189 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 355/708] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:34:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 415 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:10.976925 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670933cb-943a-4214-9ea6-6bf796a67aeb None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:11.024879 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:11.123143 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a109037-a55e-4664-8b19-69f0012d09ea None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:11.147078 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670933cb-943a-4214-9ea6-6bf796a67aeb None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:11.150767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:11.174934 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0030b44-45bb-48df-8710-620002aa8aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:11.175668 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0030b44-45bb-48df-8710-620002aa8aad None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:11.175972 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0030b44-45bb-48df-8710-620002aa8aad None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:11.177172 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0030b44-45bb-48df-8710-620002aa8aad None admin] PATH_INFO: `/v3/services` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:11.177776 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0030b44-45bb-48df-8710-620002aa8aad None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:11.178080 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0030b44-45bb-48df-8710-620002aa8aad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:11.179696 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0030b44-45bb-48df-8710-620002aa8aad None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:11.185176 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 354/709] 10.222.0.15 () {56 vars in 1124 bytes} [Sun Sep 11 10:34:10 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:34:11.186495 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584ab2ac-fc7c-40a1-8b1b-4d7d7f08a144 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:11.186787 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584ab2ac-fc7c-40a1-8b1b-4d7d7f08a144 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:11.187065 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584ab2ac-fc7c-40a1-8b1b-4d7d7f08a144 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:11.193016 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 355/710] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:34:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:34:11.193016 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584ab2ac-fc7c-40a1-8b1b-4d7d7f08a144 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:11.229171 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2ba8cd-ddb1-4b12-9c00-016b8fd3980c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:11.229863 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ba8cd-ddb1-4b12-9c00-016b8fd3980c None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:11.230162 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ba8cd-ddb1-4b12-9c00-016b8fd3980c None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:11.230451 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ba8cd-ddb1-4b12-9c00-016b8fd3980c None admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:11.231101 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2ba8cd-ddb1-4b12-9c00-016b8fd3980c None admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:11.231409 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2ba8cd-ddb1-4b12-9c00-016b8fd3980c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:11.234577 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2ba8cd-ddb1-4b12-9c00-016b8fd3980c None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:11.253203 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 356/711] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:34:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:11.254951 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4c3041-0be9-4b72-8ae2-4bd52cc535ba None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:11.255311 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4c3041-0be9-4b72-8ae2-4bd52cc535ba None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:11.255694 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4c3041-0be9-4b72-8ae2-4bd52cc535ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:11.283615 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-7a4c3041-0be9-4b72-8ae2-4bd52cc535ba None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:34:11.285127 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-7a4c3041-0be9-4b72-8ae2-4bd52cc535ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:34:11.343096 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584ab2ac-fc7c-40a1-8b1b-4d7d7f08a144 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:11.386235 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fdae91-62b7-447d-9886-94031d668436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:11.387000 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fdae91-62b7-447d-9886-94031d668436 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:11.387404 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fdae91-62b7-447d-9886-94031d668436 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:11.387754 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fdae91-62b7-447d-9886-94031d668436 None admin] PATH_INFO: `/v3/services` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:11.387999 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 357/712] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:34:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:34:11.388486 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fdae91-62b7-447d-9886-94031d668436 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:11.388845 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fdae91-62b7-447d-9886-94031d668436 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:11.389425 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac61e5-a81b-4932-9f5d-771cc133fdfe None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:11.389698 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac61e5-a81b-4932-9f5d-771cc133fdfe None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:11.389976 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac61e5-a81b-4932-9f5d-771cc133fdfe None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:11.390277 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fdae91-62b7-447d-9886-94031d668436 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:11.397438 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 356/713] 10.222.0.15 () {56 vars in 1124 bytes} [Sun Sep 11 10:34:11 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:11.398608 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fdae91-62b7-447d-9886-94031d668436 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:11.407679 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 358/714] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:34:11 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:34:11.410981 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7f2277-e113-45d2-82c6-a452e3decc94 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:11.411305 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7f2277-e113-45d2-82c6-a452e3decc94 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:11.411611 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7f2277-e113-45d2-82c6-a452e3decc94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:11.429441 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:11.446657 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-5d7f2277-e113-45d2-82c6-a452e3decc94 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:34:11.457632 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-5d7f2277-e113-45d2-82c6-a452e3decc94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:34:11.522638 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:11.573781 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 359/715] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:34:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 10:34:11.639046 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fdae91-62b7-447d-9886-94031d668436 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:11.685149 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b4c3e-3b04-4ca6-a039-5b5c75901ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:11.688100 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b4c3e-3b04-4ca6-a039-5b5c75901ecb None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:11.688100 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b4c3e-3b04-4ca6-a039-5b5c75901ecb None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:11.688100 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b4c3e-3b04-4ca6-a039-5b5c75901ecb None admin] PATH_INFO: `/v3/users/nova` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:11.700047 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6b4c3e-3b04-4ca6-a039-5b5c75901ecb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:11.701508 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6b4c3e-3b04-4ca6-a039-5b5c75901ecb None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:11.716657 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-3b6b4c3e-3b04-4ca6-a039-5b5c75901ecb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 11 10:34:11.716657 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:11.716657 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:11.716657 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:11.716657 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:11.716657 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:11.716657 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:11.716657 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:11.716657 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:11.716657 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:11.716657 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:11.716657 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:11.716657 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 10:34:11.716657 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 10:34:11.716657 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 10:34:11.716657 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 10:34:11.717660 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:11.717660 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:11.717660 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 10:34:11.717660 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:34:11.717660 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 10:34:11.717660 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:34:11.717660 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:11.717660 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:11.717660 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:11.717660 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:11.717660 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:11.717660 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:11.717660 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:11.717660 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:11.717660 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:11.717660 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:11.717660 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:11.717660 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:11.720822 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:11.720822 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:11.720822 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 10:34:11.720822 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 10:34:11.720822 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 10:34:11.720822 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 10:34:11.720822 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 10:34:11.720822 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 10:34:11.720822 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 11 10:34:11.720822 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:11.720822 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 357/716] 10.222.0.15 () {56 vars in 1093 bytes} [Sun Sep 11 10:34:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:11.724473 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7f2277-e113-45d2-82c6-a452e3decc94 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:11.745082 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b4c3e-3b04-4ca6-a039-5b5c75901ecb None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:11.920298 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7f2277-e113-45d2-82c6-a452e3decc94 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:11.956664 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b4c3e-3b04-4ca6-a039-5b5c75901ecb None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:11.975075 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:11.978748 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02767b58-3e09-4ccc-a5ba-23a147ae18e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:11.979710 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02767b58-3e09-4ccc-a5ba-23a147ae18e1 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:11.980227 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02767b58-3e09-4ccc-a5ba-23a147ae18e1 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:11.980647 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02767b58-3e09-4ccc-a5ba-23a147ae18e1 None admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:11.981960 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02767b58-3e09-4ccc-a5ba-23a147ae18e1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:11.982449 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02767b58-3e09-4ccc-a5ba-23a147ae18e1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:11.984588 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02767b58-3e09-4ccc-a5ba-23a147ae18e1 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:11.989528 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:12.016707 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e1d905-13d4-4788-95cf-d13223e94ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:12.017719 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1d905-13d4-4788-95cf-d13223e94ecf None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:12.018230 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1d905-13d4-4788-95cf-d13223e94ecf None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:12.018714 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1d905-13d4-4788-95cf-d13223e94ecf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:12.029033 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 360/717] 10.222.0.15 () {56 vars in 1133 bytes} [Sun Sep 11 10:34:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:12.031777 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02767b58-3e09-4ccc-a5ba-23a147ae18e1 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:12.044685 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e1d905-13d4-4788-95cf-d13223e94ecf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:34:12.050337 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-a6e1d905-13d4-4788-95cf-d13223e94ecf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 10:34:12.050337 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:12.050337 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:12.050337 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:12.050337 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:12.050337 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:12.050337 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:12.050337 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:12.050337 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:12.050337 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:12.050337 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:12.050337 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:12.050337 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:34:12.050337 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:34:12.050337 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:34:12.050337 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:34:12.051273 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:34:12.051273 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:34:12.051273 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:34:12.051273 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:34:12.051273 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:12.051273 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:12.051273 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:12.051273 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:12.051273 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:12.051273 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:12.051273 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:12.051273 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:12.051273 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:12.051273 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:12.051273 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:12.051273 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:12.052181 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:12.052181 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:12.052181 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:12.052181 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:12.052181 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:34:12.052181 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:34:12.052181 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:34:12.052181 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:34:12.052181 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:34:12.052181 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:34:12.052181 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 10:34:12.052181 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:12.063132 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 358/718] 10.222.0.15 () {56 vars in 1096 bytes} [Sun Sep 11 10:34:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:12.064308 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e1d905-13d4-4788-95cf-d13223e94ecf None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:12.258775 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02767b58-3e09-4ccc-a5ba-23a147ae18e1 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:12.280477 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e1d905-13d4-4788-95cf-d13223e94ecf None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:12.338122 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999ad55b-6f32-4421-92bb-2b0859b7773c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:12.339010 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999ad55b-6f32-4421-92bb-2b0859b7773c None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:12.340787 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999ad55b-6f32-4421-92bb-2b0859b7773c None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:12.341073 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999ad55b-6f32-4421-92bb-2b0859b7773c None admin] PATH_INFO: `/v3/roles/service` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:12.358139 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13a6dda-e721-432b-aca7-f858f5b6b843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:12.358759 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13a6dda-e721-432b-aca7-f858f5b6b843 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:12.359048 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13a6dda-e721-432b-aca7-f858f5b6b843 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:12.362022 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13a6dda-e721-432b-aca7-f858f5b6b843 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:12.364020 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999ad55b-6f32-4421-92bb-2b0859b7773c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:34:12.371456 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-999ad55b-6f32-4421-92bb-2b0859b7773c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 10:34:12.371456 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:12.371456 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:12.371456 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:12.371456 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:12.371456 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:12.371456 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:12.371456 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:12.371456 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:12.371456 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:12.371456 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:12.371456 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:12.371456 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:34:12.371456 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:34:12.371456 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:34:12.371456 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:34:12.372432 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:34:12.372432 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:34:12.372432 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:34:12.372432 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:34:12.372432 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:12.372432 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:12.372432 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:12.372432 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:12.372432 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:12.372432 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:12.372432 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:12.372432 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:12.372432 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:12.372432 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:12.372432 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:12.372432 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:12.373297 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:12.373297 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:12.373297 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:12.373297 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:12.373297 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:34:12.373297 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:34:12.373297 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:34:12.373297 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:34:12.373297 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:34:12.373297 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:34:12.373297 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 10:34:12.373297 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:12.380105 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 361/719] 10.222.0.15 () {56 vars in 1102 bytes} [Sun Sep 11 10:34:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:12.381067 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999ad55b-6f32-4421-92bb-2b0859b7773c None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:12.435863 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:12.494322 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:12.521168 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13a6dda-e721-432b-aca7-f858f5b6b843 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:12.530467 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13a6dda-e721-432b-aca7-f858f5b6b843 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:12.563639 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999ad55b-6f32-4421-92bb-2b0859b7773c None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:12.614332 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9668dafe-1974-4ff8-a6c8-170d258bf3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:12.614954 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9668dafe-1974-4ff8-a6c8-170d258bf3de None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:12.615238 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9668dafe-1974-4ff8-a6c8-170d258bf3de None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:12.615545 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9668dafe-1974-4ff8-a6c8-170d258bf3de None admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:12.617036 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9668dafe-1974-4ff8-a6c8-170d258bf3de None admin] RBAC: Adding query filter params (name=admin) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:12.617363 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9668dafe-1974-4ff8-a6c8-170d258bf3de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:12.618733 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9668dafe-1974-4ff8-a6c8-170d258bf3de None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:12.630945 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 362/720] 10.222.0.15 () {56 vars in 1099 bytes} [Sun Sep 11 10:34:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:12.637008 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9668dafe-1974-4ff8-a6c8-170d258bf3de None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:12.719342 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 359/721] 10.222.0.15 () {58 vars in 1282 bytes} [Sun Sep 11 10:34:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 651 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 10:34:12.720574 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13a6dda-e721-432b-aca7-f858f5b6b843 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:12.762376 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9668dafe-1974-4ff8-a6c8-170d258bf3de None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:12.779454 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:12.807158 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8ad57f-6665-43fa-9aa0-af4662dce731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:12.808034 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8ad57f-6665-43fa-9aa0-af4662dce731 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:12.808454 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8ad57f-6665-43fa-9aa0-af4662dce731 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:12.808881 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8ad57f-6665-43fa-9aa0-af4662dce731 None admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:12.810330 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8ad57f-6665-43fa-9aa0-af4662dce731 None admin] RBAC: Adding query filter params (name=service) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:12.811048 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8ad57f-6665-43fa-9aa0-af4662dce731 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:12.812620 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8ad57f-6665-43fa-9aa0-af4662dce731 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:12.824444 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 363/722] 10.222.0.15 () {56 vars in 1103 bytes} [Sun Sep 11 10:34:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:12.825711 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8ad57f-6665-43fa-9aa0-af4662dce731 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:12.840048 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:12.929585 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13a6dda-e721-432b-aca7-f858f5b6b843 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:12.955452 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8ad57f-6665-43fa-9aa0-af4662dce731 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:12.990002 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6720eb-3ded-4090-a6a2-4815743f2a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:12.990688 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6720eb-3ded-4090-a6a2-4815743f2a45 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:12.991000 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6720eb-3ded-4090-a6a2-4815743f2a45 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:12.991333 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6720eb-3ded-4090-a6a2-4815743f2a45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:13.002111 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6720eb-3ded-4090-a6a2-4815743f2a45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:13.002530 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f87b62-ed5a-4fbe-9682-3232f4c4119e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:13.003177 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f87b62-ed5a-4fbe-9682-3232f4c4119e None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:13.003485 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f87b62-ed5a-4fbe-9682-3232f4c4119e None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:13.003586 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6720eb-3ded-4090-a6a2-4815743f2a45 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:13.003765 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f87b62-ed5a-4fbe-9682-3232f4c4119e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:13.018886 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f87b62-ed5a-4fbe-9682-3232f4c4119e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:13.019823 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-ee6720eb-3ded-4090-a6a2-4815743f2a45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:13.019823 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:13.019823 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:13.019823 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:13.019823 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:13.019823 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:13.019823 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:13.019823 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:13.019823 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:13.019823 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:13.019823 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:13.019823 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:13.019823 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:34:13.019823 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:34:13.019823 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:34:13.019823 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:34:13.020760 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:13.020760 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:13.020760 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:13.020760 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:13.020760 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:13.020760 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:13.020760 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:13.020760 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:13.020760 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:13.020760 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:13.020760 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:13.020760 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:13.020760 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:13.020760 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:13.020760 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:13.020760 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:13.020760 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:34:13.021660 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:34:13.021660 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:13.021660 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:13.021841 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f87b62-ed5a-4fbe-9682-3232f4c4119e None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:13.025791 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 360/723] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:34:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:13.026659 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6720eb-3ded-4090-a6a2-4815743f2a45 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:13.045300 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-c1f87b62-ed5a-4fbe-9682-3232f4c4119e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:13.045300 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:13.045300 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:13.045300 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:13.045300 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:13.045300 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:13.045300 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:13.045300 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:13.045300 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:13.045300 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:13.045300 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:13.045300 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:13.045300 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:34:13.045300 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:34:13.045300 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:34:13.045300 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:34:13.046561 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:13.046561 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:13.046561 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:13.046561 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:13.046561 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:13.046561 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:13.046561 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:13.046561 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:13.046561 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:13.046561 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:13.046561 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:13.046561 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:13.046561 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:13.046561 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:13.046561 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:13.046561 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:13.046561 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:34:13.056702 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:34:13.056702 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:13.056702 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:13.056702 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 364/724] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:34:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:13.056702 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a43e0ed-8b45-4cd4-8b9c-783ff3acc0d0 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:13.056702 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a43e0ed-8b45-4cd4-8b9c-783ff3acc0d0 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:13.056702 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a43e0ed-8b45-4cd4-8b9c-783ff3acc0d0 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:13.056702 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 365/725] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:34:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:34:13.056702 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a43e0ed-8b45-4cd4-8b9c-783ff3acc0d0 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:13.086753 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:13.171282 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6720eb-3ded-4090-a6a2-4815743f2a45 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:13.191643 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a43e0ed-8b45-4cd4-8b9c-783ff3acc0d0 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:13.193228 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:13.210848 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8eb730-8d02-478c-862f-80af30b7ea9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:13.211515 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8eb730-8d02-478c-862f-80af30b7ea9c None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:13.211798 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8eb730-8d02-478c-862f-80af30b7ea9c None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:13.212073 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8eb730-8d02-478c-862f-80af30b7ea9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:13.244058 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02508b78-eada-4775-9d3b-f14867a342db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:13.245089 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02508b78-eada-4775-9d3b-f14867a342db None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:13.245596 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02508b78-eada-4775-9d3b-f14867a342db None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:13.246083 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02508b78-eada-4775-9d3b-f14867a342db None admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:13.246955 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02508b78-eada-4775-9d3b-f14867a342db None admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:13.247510 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02508b78-eada-4775-9d3b-f14867a342db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:13.249140 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02508b78-eada-4775-9d3b-f14867a342db None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:13.270093 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 366/726] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:34:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:13.271501 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02508b78-eada-4775-9d3b-f14867a342db None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:13.353023 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8eb730-8d02-478c-862f-80af30b7ea9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:13.354470 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8eb730-8d02-478c-862f-80af30b7ea9c None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:13.370767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-7c8eb730-8d02-478c-862f-80af30b7ea9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:13.370767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:13.370767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:13.370767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:13.370767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:13.370767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:13.370767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:13.370767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:13.370767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:13.370767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:13.370767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:13.370767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:13.370767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:34:13.370767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:34:13.370767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:34:13.370767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:34:13.371788 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:13.371788 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:13.371788 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:13.371788 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:13.371788 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:13.371788 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:13.371788 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:13.371788 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:13.371788 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:13.371788 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:13.371788 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:13.371788 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:13.371788 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:13.371788 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:13.371788 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:13.371788 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:13.371788 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:34:13.372687 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:34:13.372687 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:13.372687 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:13.376759 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 361/727] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:34:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:13.378046 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81a522-eb9e-4204-b080-8d60137e6ecf None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:13.378327 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81a522-eb9e-4204-b080-8d60137e6ecf None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:13.378590 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81a522-eb9e-4204-b080-8d60137e6ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:13.386219 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:13.401546 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02508b78-eada-4775-9d3b-f14867a342db None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:13.402618 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-0d81a522-eb9e-4204-b080-8d60137e6ecf None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:34:13.403823 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-0d81a522-eb9e-4204-b080-8d60137e6ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:34:13.443728 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4de567-13ca-4ce7-9b07-5787186ed07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:13.444692 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4de567-13ca-4ce7-9b07-5787186ed07e None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:13.445152 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4de567-13ca-4ce7-9b07-5787186ed07e None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:13.445431 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4de567-13ca-4ce7-9b07-5787186ed07e None admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:13.446071 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4de567-13ca-4ce7-9b07-5787186ed07e None admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:13.446373 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4de567-13ca-4ce7-9b07-5787186ed07e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:13.447805 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4de567-13ca-4ce7-9b07-5787186ed07e None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:13.462887 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 367/728] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:34:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:13.463907 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4de567-13ca-4ce7-9b07-5787186ed07e None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:13.494408 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 362/729] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:34:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:34:13.495746 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d81a522-eb9e-4204-b080-8d60137e6ecf None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:13.634979 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4de567-13ca-4ce7-9b07-5787186ed07e None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:13.671824 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d81a522-eb9e-4204-b080-8d60137e6ecf None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:13.677798 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:13.706563 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf3f349-55b8-4932-9244-47d89972555e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:13.707575 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf3f349-55b8-4932-9244-47d89972555e None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:13.708837 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf3f349-55b8-4932-9244-47d89972555e None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:13.709333 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf3f349-55b8-4932-9244-47d89972555e None admin] PATH_INFO: `/v3/users/swift` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:13.721591 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9892d5a-f96f-40f5-b208-cfce00479389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:13.722241 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9892d5a-f96f-40f5-b208-cfce00479389 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:13.722518 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9892d5a-f96f-40f5-b208-cfce00479389 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:13.722734 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf3f349-55b8-4932-9244-47d89972555e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:13.722845 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9892d5a-f96f-40f5-b208-cfce00479389 None admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:13.723468 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9892d5a-f96f-40f5-b208-cfce00479389 None admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:13.724810 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf3f349-55b8-4932-9244-47d89972555e None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:13.727410 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9892d5a-f96f-40f5-b208-cfce00479389 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:13.729340 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9892d5a-f96f-40f5-b208-cfce00479389 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:13.738785 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-edf3f349-55b8-4932-9244-47d89972555e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 11 10:34:13.738785 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:13.738785 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:13.738785 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:13.738785 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:13.738785 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:13.738785 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:13.738785 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:13.738785 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:13.738785 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:13.738785 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:13.738785 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:13.738785 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 10:34:13.738785 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 10:34:13.738785 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 10:34:13.738785 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 10:34:13.739827 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:13.739827 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:13.739827 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 10:34:13.739827 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:34:13.739827 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 10:34:13.739827 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:34:13.739827 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:13.739827 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:13.739827 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:13.739827 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:13.739827 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:13.739827 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:13.739827 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:13.739827 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:13.739827 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:13.739827 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:13.739827 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:13.739827 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:13.740789 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:13.740789 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:13.740789 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 10:34:13.740789 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 10:34:13.740789 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 10:34:13.740789 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 10:34:13.740789 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 10:34:13.740789 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 10:34:13.740789 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 11 10:34:13.740789 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:13.749859 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 368/730] 10.222.0.15 () {56 vars in 1096 bytes} [Sun Sep 11 10:34:13 2022] GET /identity/v3/users/swift => generated 83 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:13.752064 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf3f349-55b8-4932-9244-47d89972555e None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:13.752247 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 363/731] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:34:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:13.753300 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9892d5a-f96f-40f5-b208-cfce00479389 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:13.766832 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:13.880126 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf3f349-55b8-4932-9244-47d89972555e None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:13.884798 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9892d5a-f96f-40f5-b208-cfce00479389 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:13.917965 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3813cff1-da35-4358-a6a2-2ee275ee77f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:13.918531 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3813cff1-da35-4358-a6a2-2ee275ee77f6 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:13.918815 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3813cff1-da35-4358-a6a2-2ee275ee77f6 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:13.919088 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3813cff1-da35-4358-a6a2-2ee275ee77f6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:13.922685 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d085617-c23b-4b13-8a61-c6c3e04a7b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:13.923615 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d085617-c23b-4b13-8a61-c6c3e04a7b43 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:13.923892 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d085617-c23b-4b13-8a61-c6c3e04a7b43 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:13.924158 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d085617-c23b-4b13-8a61-c6c3e04a7b43 None admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:13.924879 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d085617-c23b-4b13-8a61-c6c3e04a7b43 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:13.925326 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d085617-c23b-4b13-8a61-c6c3e04a7b43 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:13.926229 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3813cff1-da35-4358-a6a2-2ee275ee77f6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:13.926656 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d085617-c23b-4b13-8a61-c6c3e04a7b43 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:13.927688 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3813cff1-da35-4358-a6a2-2ee275ee77f6 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:13.941405 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-3813cff1-da35-4358-a6a2-2ee275ee77f6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:34:13.941405 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:13.941405 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:13.941405 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:13.941405 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:13.941405 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:13.941405 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:13.941405 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:13.941405 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:13.941405 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:13.941405 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:13.941405 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:13.941405 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 10:34:13.941405 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 10:34:13.941405 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 10:34:13.942338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 10:34:13.942338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:13.942338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:13.942338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:13.942338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:13.942338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:13.942338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:13.942338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:13.942338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:13.942338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:13.942338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:13.942338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:13.942338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:13.942338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:13.942338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:13.942338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:13.942338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:13.943227 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 10:34:13.943227 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 10:34:13.943227 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 10:34:13.943227 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 10:34:13.943227 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 10:34:13.943227 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 10:34:13.943227 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:34:13.943227 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:13.943227 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 369/732] 10.222.0.15 () {56 vars in 1111 bytes} [Sun Sep 11 10:34:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:13.944483 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3813cff1-da35-4358-a6a2-2ee275ee77f6 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:13.950977 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 364/733] 10.222.0.15 () {56 vars in 1135 bytes} [Sun Sep 11 10:34:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:13.950977 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d085617-c23b-4b13-8a61-c6c3e04a7b43 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:13.973937 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:14.025533 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:14.060671 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d085617-c23b-4b13-8a61-c6c3e04a7b43 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:14.065929 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3813cff1-da35-4358-a6a2-2ee275ee77f6 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:14.097758 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d52e557-e8ab-46cf-9b27-2447d02675b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:14.098374 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d52e557-e8ab-46cf-9b27-2447d02675b1 None admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:14.098696 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d52e557-e8ab-46cf-9b27-2447d02675b1 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:14.098924 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d52e557-e8ab-46cf-9b27-2447d02675b1 None admin] PATH_INFO: `/v3/services` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:14.099609 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d52e557-e8ab-46cf-9b27-2447d02675b1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:14.101169 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d52e557-e8ab-46cf-9b27-2447d02675b1 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:14.101496 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed187acc-c4d9-45f9-989a-a8a1df0391f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:14.102465 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed187acc-c4d9-45f9-989a-a8a1df0391f3 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:14.102822 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed187acc-c4d9-45f9-989a-a8a1df0391f3 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:14.103195 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed187acc-c4d9-45f9-989a-a8a1df0391f3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:14.111598 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed187acc-c4d9-45f9-989a-a8a1df0391f3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:14.113311 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed187acc-c4d9-45f9-989a-a8a1df0391f3 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:14.119456 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 365/734] 10.222.0.15 () {60 vars in 1141 bytes} [Sun Sep 11 10:34:13 2022] POST /identity/v3/services => generated 240 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:34:14.120776 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc378d-e8fe-4434-9dcd-14e2913b4673 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:14.121058 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc378d-e8fe-4434-9dcd-14e2913b4673 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:14.121371 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc378d-e8fe-4434-9dcd-14e2913b4673 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:14.122913 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 366/735] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:34:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:34:14.123704 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffc378d-e8fe-4434-9dcd-14e2913b4673 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:14.131945 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-ed187acc-c4d9-45f9-989a-a8a1df0391f3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 10:34:14.131945 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:14.131945 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:14.131945 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:14.131945 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:14.131945 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:14.131945 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:14.131945 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:14.131945 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:14.131945 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:14.131945 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:14.131945 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:14.131945 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 10:34:14.131945 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 10:34:14.131945 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 10:34:14.131945 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 10:34:14.133019 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:14.133019 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:14.133019 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 10:34:14.133019 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:34:14.133019 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 10:34:14.133019 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:34:14.133019 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:14.133019 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:14.133019 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:14.133019 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:14.133019 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:14.133019 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:14.133019 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:14.133019 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:14.133019 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:14.133019 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:14.133019 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:14.133019 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:14.133979 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:14.133979 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:14.133979 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 10:34:14.133979 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 10:34:14.133979 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 10:34:14.133979 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 10:34:14.133979 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 10:34:14.133979 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 10:34:14.133979 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 10:34:14.133979 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:14.138149 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 370/736] 10.222.0.15 () {56 vars in 1117 bytes} [Sun Sep 11 10:34:13 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:14.139150 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed187acc-c4d9-45f9-989a-a8a1df0391f3 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:14.238498 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:14.279167 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffc378d-e8fe-4434-9dcd-14e2913b4673 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:14.284403 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed187acc-c4d9-45f9-989a-a8a1df0391f3 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:14.330383 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:14.342409 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989a432d-a886-496c-913e-f62927f557e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:14.343121 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989a432d-a886-496c-913e-f62927f557e0 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:14.343524 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989a432d-a886-496c-913e-f62927f557e0 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:14.343919 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989a432d-a886-496c-913e-f62927f557e0 None admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:14.344626 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989a432d-a886-496c-913e-f62927f557e0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:14.344937 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989a432d-a886-496c-913e-f62927f557e0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:14.346367 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989a432d-a886-496c-913e-f62927f557e0 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:14.348193 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00712f29-050c-46fa-b03e-32b1d689285b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:14.348813 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00712f29-050c-46fa-b03e-32b1d689285b None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:14.349094 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00712f29-050c-46fa-b03e-32b1d689285b None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:14.349362 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00712f29-050c-46fa-b03e-32b1d689285b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:14.360506 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 367/737] 10.222.0.15 () {56 vars in 1148 bytes} [Sun Sep 11 10:34:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:14.361881 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a75652-1df5-4c4c-9894-4ca7fdaae8f7 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:14.362183 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a75652-1df5-4c4c-9894-4ca7fdaae8f7 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:14.362437 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a75652-1df5-4c4c-9894-4ca7fdaae8f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:14.389980 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-d7a75652-1df5-4c4c-9894-4ca7fdaae8f7 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:34:14.391664 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-d7a75652-1df5-4c4c-9894-4ca7fdaae8f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:34:14.444733 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00712f29-050c-46fa-b03e-32b1d689285b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:14.446524 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00712f29-050c-46fa-b03e-32b1d689285b None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:14.491824 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 368/738] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:34:14 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 10:34:14.493084 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a75652-1df5-4c4c-9894-4ca7fdaae8f7 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:14.619237 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:14.634801 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 371/739] 10.222.0.15 () {58 vars in 1282 bytes} [Sun Sep 11 10:34:14 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 10:34:14.635705 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00712f29-050c-46fa-b03e-32b1d689285b None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:14.682311 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a75652-1df5-4c4c-9894-4ca7fdaae8f7 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:14.687035 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:14.734862 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c3c198-f07c-477b-9bad-61d50b56c550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:14.735561 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c3c198-f07c-477b-9bad-61d50b56c550 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:14.735858 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c3c198-f07c-477b-9bad-61d50b56c550 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:14.736098 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c3c198-f07c-477b-9bad-61d50b56c550 None admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:14.736828 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c3c198-f07c-477b-9bad-61d50b56c550 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:14.737135 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c3c198-f07c-477b-9bad-61d50b56c550 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:14.739063 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c3c198-f07c-477b-9bad-61d50b56c550 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:14.768503 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 369/740] 10.222.0.15 () {56 vars in 1149 bytes} [Sun Sep 11 10:34:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:14.768503 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c3c198-f07c-477b-9bad-61d50b56c550 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:14.805956 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00712f29-050c-46fa-b03e-32b1d689285b None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:14.851731 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea00ede-1a15-40ee-9e9d-2275b400d046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:14.852951 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea00ede-1a15-40ee-9e9d-2275b400d046 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:14.853433 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea00ede-1a15-40ee-9e9d-2275b400d046 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:14.853924 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea00ede-1a15-40ee-9e9d-2275b400d046 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:14.854847 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea00ede-1a15-40ee-9e9d-2275b400d046 None admin] RBAC: Adding query filter params (role.id=2cb7d73f0ea8414296367573f6902340, scope.project.id=5e1f845f99f04202b83621cb65e0adc2, user.id=2bd5356829b6454993bab9bad2f79fed) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:14.855382 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea00ede-1a15-40ee-9e9d-2275b400d046 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:14.857039 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea00ede-1a15-40ee-9e9d-2275b400d046 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:14.863726 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 372/741] 10.222.0.15 () {56 vars in 1374 bytes} [Sun Sep 11 10:34:14 2022] GET /identity/v3/role_assignments?user.id=2bd5356829b6454993bab9bad2f79fed&scope.project.id=5e1f845f99f04202b83621cb65e0adc2&role.id=2cb7d73f0ea8414296367573f6902340 => generated 259 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:14.864799 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea00ede-1a15-40ee-9e9d-2275b400d046 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:14.894413 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c3c198-f07c-477b-9bad-61d50b56c550 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:14.931796 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:14.951826 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b33c99-b004-4976-9bb7-f2cced7bd9aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:14.953031 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b33c99-b004-4976-9bb7-f2cced7bd9aa None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:14.953529 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b33c99-b004-4976-9bb7-f2cced7bd9aa None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:14.954053 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b33c99-b004-4976-9bb7-f2cced7bd9aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:14.967308 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b33c99-b004-4976-9bb7-f2cced7bd9aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:14.969602 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b33c99-b004-4976-9bb7-f2cced7bd9aa None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:14.996178 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-37b33c99-b004-4976-9bb7-f2cced7bd9aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:14.996178 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:14.996178 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:14.996178 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:14.996178 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:14.996178 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:14.996178 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:14.996178 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:14.996178 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:14.996178 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:14.996178 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:14.996178 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:14.996178 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:34:14.996178 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:34:14.996178 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:34:14.996178 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:34:14.997190 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:14.997190 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:14.997190 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:14.997190 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:14.997190 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:14.997190 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:14.997190 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:14.997190 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:14.997190 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:14.997190 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:14.997190 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:14.997190 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:14.997190 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:14.997190 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:14.997190 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:14.997190 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:14.997190 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:34:14.998072 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:34:14.998072 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:14.998072 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:15.003190 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:15.003770 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 370/742] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:34:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:15.005566 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b33c99-b004-4976-9bb7-f2cced7bd9aa None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:15.014754 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea00ede-1a15-40ee-9e9d-2275b400d046 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:15.104495 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ef1db4-b456-4390-8353-3efa8d4d849e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:15.105835 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef1db4-b456-4390-8353-3efa8d4d849e None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:15.106130 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef1db4-b456-4390-8353-3efa8d4d849e None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:15.106390 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef1db4-b456-4390-8353-3efa8d4d849e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:15.197116 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ef1db4-b456-4390-8353-3efa8d4d849e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:15.198583 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ef1db4-b456-4390-8353-3efa8d4d849e None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:15.205653 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b33c99-b004-4976-9bb7-f2cced7bd9aa None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:15.286479 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7c85a0-ed6e-4b9e-96ba-699a902607b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:15.287117 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7c85a0-ed6e-4b9e-96ba-699a902607b9 None admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:15.287401 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7c85a0-ed6e-4b9e-96ba-699a902607b9 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:15.287758 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7c85a0-ed6e-4b9e-96ba-699a902607b9 None admin] PATH_INFO: `/v3/services` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:15.288678 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7c85a0-ed6e-4b9e-96ba-699a902607b9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:15.290404 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7c85a0-ed6e-4b9e-96ba-699a902607b9 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:15.320949 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:15.322024 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 371/743] 10.222.0.15 () {60 vars in 1141 bytes} [Sun Sep 11 10:34:15 2022] POST /identity/v3/services => generated 251 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:34:15.323932 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7c85a0-ed6e-4b9e-96ba-699a902607b9 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:15.335537 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 373/744] 10.222.0.15 () {58 vars in 1282 bytes} [Sun Sep 11 10:34:14 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 10:34:15.342323 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ef1db4-b456-4390-8353-3efa8d4d849e None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:15.411694 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:15.570186 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ef1db4-b456-4390-8353-3efa8d4d849e None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:15.573496 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7c85a0-ed6e-4b9e-96ba-699a902607b9 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:15.639880 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c52808-d83f-4f21-adcd-a27ecf5c077c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:15.640923 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c52808-d83f-4f21-adcd-a27ecf5c077c None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:15.641643 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c52808-d83f-4f21-adcd-a27ecf5c077c None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:15.642121 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c52808-d83f-4f21-adcd-a27ecf5c077c None admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:15.642968 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c52808-d83f-4f21-adcd-a27ecf5c077c None admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:15.643415 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7e339a-16cd-4afe-b2a9-c83f5eddcc58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:15.643828 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c52808-d83f-4f21-adcd-a27ecf5c077c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:15.645160 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7e339a-16cd-4afe-b2a9-c83f5eddcc58 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:15.645445 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7e339a-16cd-4afe-b2a9-c83f5eddcc58 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:15.645718 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7e339a-16cd-4afe-b2a9-c83f5eddcc58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:15.646474 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c52808-d83f-4f21-adcd-a27ecf5c077c None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:15.661275 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7e339a-16cd-4afe-b2a9-c83f5eddcc58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:15.663054 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7e339a-16cd-4afe-b2a9-c83f5eddcc58 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:15.664289 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 372/745] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:34:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:15.673637 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c52808-d83f-4f21-adcd-a27ecf5c077c None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:15.686506 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-0f7e339a-16cd-4afe-b2a9-c83f5eddcc58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:15.686506 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:15.686506 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:15.686506 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:15.686506 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:15.686506 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:15.686506 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:15.686506 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:15.686506 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:15.686506 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:15.686506 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:15.686506 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:15.686506 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:34:15.686506 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:34:15.686506 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:34:15.686506 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:34:15.687525 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:15.687525 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:15.687525 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:15.687525 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:15.687525 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:15.687525 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:15.687525 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:15.687525 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:15.687525 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:15.687525 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:15.687525 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:15.687525 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:15.687525 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:15.687525 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:15.687525 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:15.687525 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:15.687525 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:34:15.688523 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:34:15.688523 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:15.688523 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:15.691610 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 374/746] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:34:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:15.695896 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7e339a-16cd-4afe-b2a9-c83f5eddcc58 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:15.773684 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:15.854411 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:15.868324 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7e339a-16cd-4afe-b2a9-c83f5eddcc58 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:15.908124 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c52808-d83f-4f21-adcd-a27ecf5c077c None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:15.914523 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef90d8f-0c99-4b0f-9429-703fc1578274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:15.915082 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef90d8f-0c99-4b0f-9429-703fc1578274 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:15.915360 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef90d8f-0c99-4b0f-9429-703fc1578274 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:15.915678 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef90d8f-0c99-4b0f-9429-703fc1578274 None admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:15.916317 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef90d8f-0c99-4b0f-9429-703fc1578274 None admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:15.916614 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef90d8f-0c99-4b0f-9429-703fc1578274 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:15.917987 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef90d8f-0c99-4b0f-9429-703fc1578274 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:15.934042 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 375/747] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:34:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:15.948520 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef90d8f-0c99-4b0f-9429-703fc1578274 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:16.004804 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5a4781-2a1a-45ef-82ef-ade6daea44f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:16.005699 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5a4781-2a1a-45ef-82ef-ade6daea44f8 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:16.006128 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5a4781-2a1a-45ef-82ef-ade6daea44f8 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:16.006657 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5a4781-2a1a-45ef-82ef-ade6daea44f8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:16.020429 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5a4781-2a1a-45ef-82ef-ade6daea44f8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:16.023999 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5a4781-2a1a-45ef-82ef-ade6daea44f8 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:16.055890 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-7a5a4781-2a1a-45ef-82ef-ade6daea44f8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:34:16.055890 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:16.055890 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:16.055890 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:16.055890 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:16.055890 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:16.055890 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:16.055890 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:16.055890 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:16.055890 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:16.055890 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:16.055890 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:16.055890 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 10:34:16.055890 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 10:34:16.055890 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 10:34:16.056983 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 10:34:16.056983 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:16.056983 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:16.056983 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:16.056983 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:16.056983 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:16.056983 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:16.056983 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:16.056983 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:16.056983 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:16.056983 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:16.056983 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:16.056983 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:16.056983 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:16.056983 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:16.056983 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:16.056983 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:16.057910 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 10:34:16.057910 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 10:34:16.057910 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 10:34:16.057910 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 10:34:16.057910 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 10:34:16.057910 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 10:34:16.057910 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:34:16.057910 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:16.062224 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 373/748] 10.222.0.15 () {56 vars in 1111 bytes} [Sun Sep 11 10:34:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 392 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:16.071282 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5a4781-2a1a-45ef-82ef-ade6daea44f8 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:16.134411 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef90d8f-0c99-4b0f-9429-703fc1578274 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:16.141102 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:16.194063 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b59051-478c-4903-bf50-0312946750df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:16.194951 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b59051-478c-4903-bf50-0312946750df None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:16.195247 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b59051-478c-4903-bf50-0312946750df None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:16.195546 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b59051-478c-4903-bf50-0312946750df None admin] PATH_INFO: `/v3/projects/service` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:16.205168 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b59051-478c-4903-bf50-0312946750df None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:16.207974 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b59051-478c-4903-bf50-0312946750df None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:16.231660 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-66b59051-478c-4903-bf50-0312946750df None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:34:16.231660 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:16.231660 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:16.231660 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:16.231660 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:16.231660 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:16.231660 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:16.231660 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:16.231660 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:16.231660 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:16.231660 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:16.231660 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:16.231660 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 10:34:16.231660 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 10:34:16.231660 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 10:34:16.232665 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 10:34:16.232665 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:16.232665 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:16.232665 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:16.232665 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:16.232665 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:16.232665 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:16.232665 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:16.232665 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:16.232665 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:16.232665 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:16.232665 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:16.232665 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:16.232665 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:16.232665 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:16.232665 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:16.232665 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:16.233576 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 10:34:16.233576 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 10:34:16.233576 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 10:34:16.233576 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 10:34:16.233576 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 10:34:16.233576 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 10:34:16.233576 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:34:16.233576 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:16.234316 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 376/749] 10.222.0.15 () {56 vars in 1111 bytes} [Sun Sep 11 10:34:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:16.242789 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b59051-478c-4903-bf50-0312946750df None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:16.324193 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5a4781-2a1a-45ef-82ef-ade6daea44f8 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:16.377902 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dac1c0-0595-4605-a454-21813080bb48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:16.378957 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dac1c0-0595-4605-a454-21813080bb48 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:16.379471 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dac1c0-0595-4605-a454-21813080bb48 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:16.379952 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dac1c0-0595-4605-a454-21813080bb48 None admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:16.380871 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dac1c0-0595-4605-a454-21813080bb48 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:16.381506 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dac1c0-0595-4605-a454-21813080bb48 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:16.383896 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dac1c0-0595-4605-a454-21813080bb48 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:16.416869 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 374/750] 10.222.0.15 () {56 vars in 1148 bytes} [Sun Sep 11 10:34:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:34:16.424604 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dac1c0-0595-4605-a454-21813080bb48 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:16.460440 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:16.485990 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b59051-478c-4903-bf50-0312946750df None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:16.575747 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2db7487-8dce-45f6-a3cc-dd8e9f31fe1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:16.576717 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2db7487-8dce-45f6-a3cc-dd8e9f31fe1a None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:16.577201 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2db7487-8dce-45f6-a3cc-dd8e9f31fe1a None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:16.577700 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2db7487-8dce-45f6-a3cc-dd8e9f31fe1a None admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:16.578680 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2db7487-8dce-45f6-a3cc-dd8e9f31fe1a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:16.579203 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2db7487-8dce-45f6-a3cc-dd8e9f31fe1a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:16.580950 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2db7487-8dce-45f6-a3cc-dd8e9f31fe1a None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:16.598846 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 377/751] 10.222.0.15 () {56 vars in 1148 bytes} [Sun Sep 11 10:34:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:16.600659 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fa439f-fecf-4e8a-9e24-b4147579cd19 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:16.600998 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fa439f-fecf-4e8a-9e24-b4147579cd19 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:16.601277 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fa439f-fecf-4e8a-9e24-b4147579cd19 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:16.604097 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 378/752] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:34:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:34:16.607953 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fa439f-fecf-4e8a-9e24-b4147579cd19 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:16.640919 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dac1c0-0595-4605-a454-21813080bb48 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:16.647056 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:16.707652 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e76ea5d-50ae-4a11-a117-d58e80a5c07c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:16.708272 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e76ea5d-50ae-4a11-a117-d58e80a5c07c None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:16.708554 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e76ea5d-50ae-4a11-a117-d58e80a5c07c None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:16.708824 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e76ea5d-50ae-4a11-a117-d58e80a5c07c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:16.709640 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e76ea5d-50ae-4a11-a117-d58e80a5c07c None admin] RBAC: Adding query filter params (role.id=2cb7d73f0ea8414296367573f6902340, scope.project.id=5e1f845f99f04202b83621cb65e0adc2, user.id=cc7ca566e66a45b29e3d02f47bf84981) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:16.714185 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e76ea5d-50ae-4a11-a117-d58e80a5c07c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:16.717103 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e76ea5d-50ae-4a11-a117-d58e80a5c07c None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:16.736721 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 375/753] 10.222.0.15 () {56 vars in 1374 bytes} [Sun Sep 11 10:34:16 2022] GET /identity/v3/role_assignments?user.id=cc7ca566e66a45b29e3d02f47bf84981&scope.project.id=5e1f845f99f04202b83621cb65e0adc2&role.id=2cb7d73f0ea8414296367573f6902340 => generated 259 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:16.738222 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a622b-5cff-494a-b45e-037d8c6daddc None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:16.738509 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a622b-5cff-494a-b45e-037d8c6daddc None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:16.738806 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a622b-5cff-494a-b45e-037d8c6daddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:16.790292 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-782a622b-5cff-494a-b45e-037d8c6daddc None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:34:16.791534 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-782a622b-5cff-494a-b45e-037d8c6daddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:34:16.813521 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fa439f-fecf-4e8a-9e24-b4147579cd19 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:16.900704 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0af109-d920-47ba-b8b8-b44346384892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:16.903447 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0af109-d920-47ba-b8b8-b44346384892 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:16.903447 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0af109-d920-47ba-b8b8-b44346384892 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:16.903447 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0af109-d920-47ba-b8b8-b44346384892 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:16.903447 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0af109-d920-47ba-b8b8-b44346384892 None admin] RBAC: Adding query filter params (role.id=3cb7ebbc707d453586b3d0f4ab3cf103, scope.project.id=5e1f845f99f04202b83621cb65e0adc2, user.id=bb44e7aef0c7436d93a7233307b5f095) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:16.903447 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0af109-d920-47ba-b8b8-b44346384892 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:16.905950 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0af109-d920-47ba-b8b8-b44346384892 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:16.926515 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 379/754] 10.222.0.15 () {56 vars in 1374 bytes} [Sun Sep 11 10:34:16 2022] GET /identity/v3/role_assignments?user.id=bb44e7aef0c7436d93a7233307b5f095&scope.project.id=5e1f845f99f04202b83621cb65e0adc2&role.id=3cb7ebbc707d453586b3d0f4ab3cf103 => generated 259 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:16.962464 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 376/755] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:34:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 10:34:17.308602 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0af109-d920-47ba-b8b8-b44346384892 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:17.494906 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:17.511872 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0af109-d920-47ba-b8b8-b44346384892 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:17.580967 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c9479e-6de0-4243-b0f9-e5edff649fdb None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:17.581447 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c9479e-6de0-4243-b0f9-e5edff649fdb None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:17.581950 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c9479e-6de0-4243-b0f9-e5edff649fdb None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:17.589553 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 377/756] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:34:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:34:17.591082 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ccca7-8844-48d5-bd9a-256a4c97a0cf None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:17.591397 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ccca7-8844-48d5-bd9a-256a4c97a0cf None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:17.591793 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ccca7-8844-48d5-bd9a-256a4c97a0cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:17.601185 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1462731-5e55-4852-80e5-afba15867b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:17.601366 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1462731-5e55-4852-80e5-afba15867b4c None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:17.601738 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1462731-5e55-4852-80e5-afba15867b4c None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:17.601914 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1462731-5e55-4852-80e5-afba15867b4c None admin] PATH_INFO: `/v3/services/network` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:17.602767 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1462731-5e55-4852-80e5-afba15867b4c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:17.604214 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1462731-5e55-4852-80e5-afba15867b4c None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:17.607160 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:17.614440 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-d23ccca7-8844-48d5-bd9a-256a4c97a0cf None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:34:17.615937 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-d23ccca7-8844-48d5-bd9a-256a4c97a0cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:34:17.624868 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-a1462731-5e55-4852-80e5-afba15867b4c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 11 10:34:17.624868 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:17.624868 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 10:34:17.624868 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 10:34:17.624868 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 10:34:17.624868 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 10:34:17.624868 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 10:34:17.624868 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 10:34:17.624868 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 11 10:34:17.624868 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:17.624868 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 10:34:17.624868 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:17.624868 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:17.624868 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:17.624868 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:17.624868 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:17.624868 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:17.625907 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:17.625907 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:17.625907 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:17.625907 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:17.625907 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:17.625907 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:17.625907 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 10:34:17.625907 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 10:34:17.625907 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 10:34:17.625907 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 10:34:17.625907 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:17.625907 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:17.625907 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:17.625907 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:17.625907 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:17.625907 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:17.627939 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:17.627939 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:17.627939 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:17.627939 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:17.627939 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:17.627939 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:17.627939 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:17.627939 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:17.627939 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:17.627939 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:17.627939 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 10:34:17.627939 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 10:34:17.627939 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 11 10:34:17.627939 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:17.627939 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 380/757] 10.222.0.15 () {56 vars in 1111 bytes} [Sun Sep 11 10:34:17 2022] GET /identity/v3/services/network => generated 88 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:17.634116 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1462731-5e55-4852-80e5-afba15867b4c None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:17.759959 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 378/758] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:34:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 10:34:17.767321 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1462731-5e55-4852-80e5-afba15867b4c None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:17.831517 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3a08dc-7b43-48d6-85d2-03b610fbc4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:17.831702 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3a08dc-7b43-48d6-85d2-03b610fbc4e9 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:17.831937 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3a08dc-7b43-48d6-85d2-03b610fbc4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:17.832233 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3a08dc-7b43-48d6-85d2-03b610fbc4e9 None admin] PATH_INFO: `/v3/services` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:17.833312 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3a08dc-7b43-48d6-85d2-03b610fbc4e9 None admin] RBAC: Adding query filter params (name=network) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:17.833312 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3a08dc-7b43-48d6-85d2-03b610fbc4e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:17.835953 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3a08dc-7b43-48d6-85d2-03b610fbc4e9 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:17.851548 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 381/759] 10.222.0.15 () {56 vars in 1112 bytes} [Sun Sep 11 10:34:17 2022] GET /identity/v3/services?name=network => generated 124 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:17.857715 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23ccca7-8844-48d5-bd9a-256a4c97a0cf None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:18.042335 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23ccca7-8844-48d5-bd9a-256a4c97a0cf None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:18.141447 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adb833f-3872-427b-83d6-64ea6ad69b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:18.142006 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adb833f-3872-427b-83d6-64ea6ad69b32 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:18.142289 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adb833f-3872-427b-83d6-64ea6ad69b32 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:18.142561 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adb833f-3872-427b-83d6-64ea6ad69b32 None admin] PATH_INFO: `/v3/services` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:18.143196 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adb833f-3872-427b-83d6-64ea6ad69b32 None admin] RBAC: Adding query filter params (type=network) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:18.143519 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adb833f-3872-427b-83d6-64ea6ad69b32 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:18.144895 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adb833f-3872-427b-83d6-64ea6ad69b32 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:18.152888 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:18.156455 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 379/760] 10.222.0.15 () {56 vars in 1112 bytes} [Sun Sep 11 10:34:17 2022] GET /identity/v3/services?type=network => generated 350 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:18.163899 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3a08dc-7b43-48d6-85d2-03b610fbc4e9 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:18.190487 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:18.200481 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adb833f-3872-427b-83d6-64ea6ad69b32 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:18.310933 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3a08dc-7b43-48d6-85d2-03b610fbc4e9 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:18.355266 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662775ae-ed82-4f0c-b1fc-243a53bf4eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:18.356162 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662775ae-ed82-4f0c-b1fc-243a53bf4eef None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:18.356606 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662775ae-ed82-4f0c-b1fc-243a53bf4eef None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:18.357026 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662775ae-ed82-4f0c-b1fc-243a53bf4eef None admin] PATH_INFO: `/v3/endpoints` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:18.357867 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662775ae-ed82-4f0c-b1fc-243a53bf4eef None admin] RBAC: Adding query filter params (interface=public, service_id=ee86803cf1384caeb74236830643b5b1, region_id=RegionOne) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:18.358433 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662775ae-ed82-4f0c-b1fc-243a53bf4eef None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:18.359980 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662775ae-ed82-4f0c-b1fc-243a53bf4eef None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:18.374379 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 382/761] 10.222.0.15 () {56 vars in 1251 bytes} [Sun Sep 11 10:34:18 2022] GET /identity/v3/endpoints?service_id=ee86803cf1384caeb74236830643b5b1&interface=public®ion_id=RegionOne => generated 194 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:18.376101 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d30c51-05dd-46f9-adc7-1fc33362ee49 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:18.376666 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d30c51-05dd-46f9-adc7-1fc33362ee49 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:18.377103 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d30c51-05dd-46f9-adc7-1fc33362ee49 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:18.378851 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 383/762] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:34:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:34:18.379722 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d30c51-05dd-46f9-adc7-1fc33362ee49 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:18.412383 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adb833f-3872-427b-83d6-64ea6ad69b32 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:18.467040 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07ecdce-1f71-404b-9368-f9feaacb4e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:18.467703 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ecdce-1f71-404b-9368-f9feaacb4e52 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:18.467984 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ecdce-1f71-404b-9368-f9feaacb4e52 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:18.468256 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ecdce-1f71-404b-9368-f9feaacb4e52 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:18.477809 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:18.490413 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07ecdce-1f71-404b-9368-f9feaacb4e52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:34:18.496409 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-e07ecdce-1f71-404b-9368-f9feaacb4e52 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 10:34:18.496409 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:18.496409 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:18.496409 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:18.496409 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:18.496409 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:18.496409 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:18.496409 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:18.496409 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:18.496409 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:18.496409 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:18.496409 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:18.496409 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:34:18.496409 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:34:18.496409 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:34:18.496409 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:34:18.497447 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:34:18.497447 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:34:18.497447 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:34:18.497447 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:34:18.497447 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:18.497447 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:18.497447 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:18.497447 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:18.497447 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:18.497447 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:18.497447 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:18.497447 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:18.497447 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:18.497447 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:18.497447 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:18.497447 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:18.498318 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:18.498318 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:18.498318 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:18.498318 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:18.498318 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:34:18.498318 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:34:18.498318 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:34:18.498318 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:34:18.498318 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:34:18.498318 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:34:18.498318 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 10:34:18.498318 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:18.501366 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 380/763] 10.222.0.15 () {56 vars in 1096 bytes} [Sun Sep 11 10:34:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:18.502662 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f42318-7f44-41a6-8da6-c7b78aba7425 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:18.502942 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f42318-7f44-41a6-8da6-c7b78aba7425 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:18.503209 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f42318-7f44-41a6-8da6-c7b78aba7425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:18.510154 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d30c51-05dd-46f9-adc7-1fc33362ee49 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:18.541069 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-f0f42318-7f44-41a6-8da6-c7b78aba7425 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:34:18.541769 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-f0f42318-7f44-41a6-8da6-c7b78aba7425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:34:18.551832 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5835c3ef-34f8-40d1-a12b-b281fd20a543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:18.552691 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5835c3ef-34f8-40d1-a12b-b281fd20a543 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:18.553172 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5835c3ef-34f8-40d1-a12b-b281fd20a543 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:18.554907 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5835c3ef-34f8-40d1-a12b-b281fd20a543 None admin] PATH_INFO: `/v3/services` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:18.555718 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5835c3ef-34f8-40d1-a12b-b281fd20a543 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:18.557655 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5835c3ef-34f8-40d1-a12b-b281fd20a543 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:18.571371 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 384/764] 10.222.0.15 () {56 vars in 1087 bytes} [Sun Sep 11 10:34:18 2022] GET /identity/v3/services => generated 772 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:18.572650 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5835c3ef-34f8-40d1-a12b-b281fd20a543 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:18.649302 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:18.674555 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 381/765] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:34:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 10:34:18.814401 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5835c3ef-34f8-40d1-a12b-b281fd20a543 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:18.862344 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6796ba43-8092-4546-a028-9d2816e48d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:18.863314 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6796ba43-8092-4546-a028-9d2816e48d9c None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:18.863884 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6796ba43-8092-4546-a028-9d2816e48d9c None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:18.864361 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6796ba43-8092-4546-a028-9d2816e48d9c None admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:18.865230 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6796ba43-8092-4546-a028-9d2816e48d9c None admin] RBAC: Adding query filter params (name=admin) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:18.865807 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6796ba43-8092-4546-a028-9d2816e48d9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:18.867675 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6796ba43-8092-4546-a028-9d2816e48d9c None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:18.877532 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 385/766] 10.222.0.15 () {56 vars in 1099 bytes} [Sun Sep 11 10:34:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:18.884712 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f42318-7f44-41a6-8da6-c7b78aba7425 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:19.009122 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f42318-7f44-41a6-8da6-c7b78aba7425 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:19.046016 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92740b8-692b-4c0b-808e-2e87fdc460cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:19.046656 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92740b8-692b-4c0b-808e-2e87fdc460cd None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:19.046929 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92740b8-692b-4c0b-808e-2e87fdc460cd None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:19.047196 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92740b8-692b-4c0b-808e-2e87fdc460cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:19.058849 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92740b8-692b-4c0b-808e-2e87fdc460cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:19.061010 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92740b8-692b-4c0b-808e-2e87fdc460cd None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:19.061125 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6796ba43-8092-4546-a028-9d2816e48d9c None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:19.077534 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-f92740b8-692b-4c0b-808e-2e87fdc460cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:19.077534 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:19.077534 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:19.077534 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:19.077534 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:19.077534 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:19.077534 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:19.077534 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:19.077534 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:19.077534 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:19.077534 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:19.077534 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:19.077534 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:34:19.077534 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:34:19.077534 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:34:19.077534 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:34:19.078543 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:19.078543 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:19.078543 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:19.078543 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:19.078543 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:19.078543 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:19.078543 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:19.078543 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:19.078543 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:19.078543 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:19.078543 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:19.078543 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:19.078543 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:19.078543 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:19.078543 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:19.078543 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:19.078543 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:34:19.083914 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:34:19.083914 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:19.083914 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:19.083914 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 382/767] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:34:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:19.085443 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:19.086437 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92740b8-692b-4c0b-808e-2e87fdc460cd None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:19.144869 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:19.217568 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92740b8-692b-4c0b-808e-2e87fdc460cd None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:19.255294 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d3b282-1265-4c48-a51b-62f406428c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:19.257090 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d3b282-1265-4c48-a51b-62f406428c4e None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:19.257546 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d3b282-1265-4c48-a51b-62f406428c4e None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:19.257963 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d3b282-1265-4c48-a51b-62f406428c4e None admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:19.258726 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d3b282-1265-4c48-a51b-62f406428c4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:19.259201 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d3b282-1265-4c48-a51b-62f406428c4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:19.261506 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d3b282-1265-4c48-a51b-62f406428c4e None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:19.274831 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 383/768] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:34:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:34:19.277994 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6796ba43-8092-4546-a028-9d2816e48d9c None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:19.286067 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d3b282-1265-4c48-a51b-62f406428c4e None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:19.376180 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29d53d5-2289-4cb8-a59c-9d2a98dad489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:19.376840 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d53d5-2289-4cb8-a59c-9d2a98dad489 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:19.377124 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d53d5-2289-4cb8-a59c-9d2a98dad489 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:19.377395 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d53d5-2289-4cb8-a59c-9d2a98dad489 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:19.377980 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29d53d5-2289-4cb8-a59c-9d2a98dad489 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:19.380567 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29d53d5-2289-4cb8-a59c-9d2a98dad489 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:19.405325 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d3b282-1265-4c48-a51b-62f406428c4e None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:19.406989 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:19.417326 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-b29d53d5-2289-4cb8-a59c-9d2a98dad489 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 11 10:34:19.417326 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:19.417326 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 10:34:19.417326 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 10:34:19.417326 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 10:34:19.417326 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 10:34:19.417326 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 10:34:19.417326 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 10:34:19.417326 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 11 10:34:19.417326 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:19.417326 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 10:34:19.417326 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:19.417326 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:19.417326 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:19.417326 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:19.417326 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:19.417326 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:19.418363 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:19.418363 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:19.418363 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:19.418363 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:19.418363 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:19.418363 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:19.418363 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 10:34:19.418363 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 10:34:19.418363 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 10:34:19.418363 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 10:34:19.418363 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:19.418363 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:19.418363 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:19.418363 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:19.418363 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:19.418363 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:19.419287 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:19.419287 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:19.419287 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:19.419287 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:19.419287 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:19.419287 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:19.419287 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:19.419287 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:19.419287 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:19.419287 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:19.419287 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 10:34:19.419287 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 10:34:19.419287 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 11 10:34:19.419287 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:19.419287 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 386/769] 10.222.0.15 () {56 vars in 1129 bytes} [Sun Sep 11 10:34:19 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:19.424211 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29d53d5-2289-4cb8-a59c-9d2a98dad489 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:19.452024 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a662fb6d-5e4d-4899-a9a1-a65fa4540c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:19.452024 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a662fb6d-5e4d-4899-a9a1-a65fa4540c63 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:19.452331 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a662fb6d-5e4d-4899-a9a1-a65fa4540c63 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:19.452702 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a662fb6d-5e4d-4899-a9a1-a65fa4540c63 None admin] PATH_INFO: `/v3/users/nova` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:19.464735 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a662fb6d-5e4d-4899-a9a1-a65fa4540c63 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:19.466865 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a662fb6d-5e4d-4899-a9a1-a65fa4540c63 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:19.499477 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-a662fb6d-5e4d-4899-a9a1-a65fa4540c63 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 11 10:34:19.499477 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:19.499477 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:19.499477 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:19.499477 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:19.499477 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:19.499477 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:19.499477 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:19.499477 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:19.499477 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:19.499477 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:19.499477 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:19.499477 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 10:34:19.499477 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 10:34:19.499477 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 10:34:19.499477 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 10:34:19.500503 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:19.500503 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:19.500503 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 10:34:19.500503 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:34:19.500503 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 10:34:19.500503 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:34:19.500503 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:19.500503 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:19.500503 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:19.500503 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:19.500503 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:19.500503 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:19.500503 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:19.500503 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:19.500503 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:19.500503 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:19.500503 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:19.500503 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:19.501446 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:19.501446 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:19.501446 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 10:34:19.501446 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 10:34:19.501446 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 10:34:19.501446 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 10:34:19.501446 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 10:34:19.501446 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 10:34:19.501446 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 11 10:34:19.501446 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:19.507471 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 384/770] 10.222.0.15 () {56 vars in 1093 bytes} [Sun Sep 11 10:34:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:19.507471 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a662fb6d-5e4d-4899-a9a1-a65fa4540c63 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:19.589922 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:19.659142 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29d53d5-2289-4cb8-a59c-9d2a98dad489 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:19.688362 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a662fb6d-5e4d-4899-a9a1-a65fa4540c63 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:19.711852 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0682dac-2452-49a2-b930-02edfc02af36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:19.712614 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0682dac-2452-49a2-b930-02edfc02af36 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:19.713002 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0682dac-2452-49a2-b930-02edfc02af36 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:19.713369 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0682dac-2452-49a2-b930-02edfc02af36 None admin] PATH_INFO: `/v3/services` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:19.714011 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0682dac-2452-49a2-b930-02edfc02af36 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:19.714724 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0682dac-2452-49a2-b930-02edfc02af36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:19.715726 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0682dac-2452-49a2-b930-02edfc02af36 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:19.721540 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 387/771] 10.222.0.15 () {56 vars in 1124 bytes} [Sun Sep 11 10:34:19 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:19.722426 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dab43e-b8e9-4f8b-a8b3-29d841c5481a None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:19.722711 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dab43e-b8e9-4f8b-a8b3-29d841c5481a None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:19.722989 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dab43e-b8e9-4f8b-a8b3-29d841c5481a None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:19.724344 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 388/772] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:34:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:34:19.725891 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0353b3f3-fcb8-44ce-aa91-dff71b8304aa None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:19.726152 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0353b3f3-fcb8-44ce-aa91-dff71b8304aa None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:19.726589 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0353b3f3-fcb8-44ce-aa91-dff71b8304aa None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:19.732352 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 389/773] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:34:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:34:19.733464 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0353b3f3-fcb8-44ce-aa91-dff71b8304aa None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:19.738919 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a410c91-8ae4-4956-a9d7-349d7dd8f369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:19.739854 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a410c91-8ae4-4956-a9d7-349d7dd8f369 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:19.740337 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a410c91-8ae4-4956-a9d7-349d7dd8f369 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:19.740725 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a410c91-8ae4-4956-a9d7-349d7dd8f369 None admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:19.741631 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a410c91-8ae4-4956-a9d7-349d7dd8f369 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:19.742141 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a410c91-8ae4-4956-a9d7-349d7dd8f369 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:19.743913 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a410c91-8ae4-4956-a9d7-349d7dd8f369 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:19.770898 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 385/774] 10.222.0.15 () {56 vars in 1133 bytes} [Sun Sep 11 10:34:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:19.770898 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87451d47-73a5-4711-b037-0ef42c38386b None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:19.770898 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87451d47-73a5-4711-b037-0ef42c38386b None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:19.770898 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87451d47-73a5-4711-b037-0ef42c38386b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:19.794349 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-87451d47-73a5-4711-b037-0ef42c38386b None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:34:19.796324 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-87451d47-73a5-4711-b037-0ef42c38386b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:34:19.815696 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:19.870053 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0353b3f3-fcb8-44ce-aa91-dff71b8304aa None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:19.878345 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 386/775] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:34:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 10:34:19.888871 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660a710f-8535-4571-82b5-bf1b46fa469d None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:19.889154 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660a710f-8535-4571-82b5-bf1b46fa469d None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:19.889426 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660a710f-8535-4571-82b5-bf1b46fa469d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:19.912687 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34b17d9-b88c-4e12-aa88-b293f8f565ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:19.913314 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b17d9-b88c-4e12-aa88-b293f8f565ba None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:19.913593 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b17d9-b88c-4e12-aa88-b293f8f565ba None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:19.913868 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b17d9-b88c-4e12-aa88-b293f8f565ba None admin] PATH_INFO: `/v3/services` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:19.914501 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34b17d9-b88c-4e12-aa88-b293f8f565ba None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:19.914807 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34b17d9-b88c-4e12-aa88-b293f8f565ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:19.918248 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34b17d9-b88c-4e12-aa88-b293f8f565ba None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:19.919756 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:19.923731 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 390/776] 10.222.0.15 () {56 vars in 1124 bytes} [Sun Sep 11 10:34:19 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:19.930128 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34b17d9-b88c-4e12-aa88-b293f8f565ba None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:19.939821 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-660a710f-8535-4571-82b5-bf1b46fa469d None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:34:19.943189 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-660a710f-8535-4571-82b5-bf1b46fa469d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:34:20.045712 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 387/777] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:34:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 10:34:20.046944 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660a710f-8535-4571-82b5-bf1b46fa469d None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:20.164365 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:20.187502 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34b17d9-b88c-4e12-aa88-b293f8f565ba None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:20.206208 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660a710f-8535-4571-82b5-bf1b46fa469d None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:20.256608 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0c0d80-9807-4549-b9d5-2b22ea2cdfce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:20.257729 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0c0d80-9807-4549-b9d5-2b22ea2cdfce None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:20.258221 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0c0d80-9807-4549-b9d5-2b22ea2cdfce None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:20.258688 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0c0d80-9807-4549-b9d5-2b22ea2cdfce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:20.271027 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f0a490-e6ac-4c0f-aaa4-04297257fa03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:20.272004 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f0a490-e6ac-4c0f-aaa4-04297257fa03 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:20.272510 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f0a490-e6ac-4c0f-aaa4-04297257fa03 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:20.272987 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f0a490-e6ac-4c0f-aaa4-04297257fa03 None admin] PATH_INFO: `/v3/endpoints` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:20.273901 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f0a490-e6ac-4c0f-aaa4-04297257fa03 None admin] RBAC: Adding query filter params (interface=public, service_id=68469aca595544c4a3f52f69cd14b94d, region_id=RegionOne) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:20.274406 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f0a490-e6ac-4c0f-aaa4-04297257fa03 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:20.276068 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f0a490-e6ac-4c0f-aaa4-04297257fa03 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:20.283700 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0c0d80-9807-4549-b9d5-2b22ea2cdfce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:20.286629 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0c0d80-9807-4549-b9d5-2b22ea2cdfce None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:20.299006 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 388/778] 10.222.0.15 () {56 vars in 1251 bytes} [Sun Sep 11 10:34:20 2022] GET /identity/v3/endpoints?service_id=68469aca595544c4a3f52f69cd14b94d&interface=public®ion_id=RegionOne => generated 194 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:20.300201 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f0a490-e6ac-4c0f-aaa4-04297257fa03 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:20.316259 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-8f0c0d80-9807-4549-b9d5-2b22ea2cdfce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:20.316259 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:20.316259 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:20.316259 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:20.316259 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:20.316259 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:20.316259 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:20.316259 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:20.316259 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:20.316259 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:20.316259 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:20.316259 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:20.316259 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:34:20.316259 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:34:20.316259 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:34:20.316259 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:34:20.317463 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:20.317463 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:20.317463 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:20.317463 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:20.317463 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:20.317463 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:20.317463 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:20.317463 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:20.317463 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:20.317463 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:20.317463 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:20.317463 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:20.317463 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:20.317463 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:20.317463 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:20.317463 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:20.317463 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:34:20.318359 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:34:20.318359 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:20.318359 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:20.318359 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 391/779] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:34:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 392 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:20.318359 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0c0d80-9807-4549-b9d5-2b22ea2cdfce None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:20.408817 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:20.456630 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f0a490-e6ac-4c0f-aaa4-04297257fa03 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:20.474183 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0c0d80-9807-4549-b9d5-2b22ea2cdfce None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:20.497006 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1249358e-a2d2-4fa1-946a-f06bdc3fd09a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:20.497652 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1249358e-a2d2-4fa1-946a-f06bdc3fd09a None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:20.497990 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1249358e-a2d2-4fa1-946a-f06bdc3fd09a None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:20.498196 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1249358e-a2d2-4fa1-946a-f06bdc3fd09a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:20.512686 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1249358e-a2d2-4fa1-946a-f06bdc3fd09a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:34:20.517897 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-1249358e-a2d2-4fa1-946a-f06bdc3fd09a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 10:34:20.517897 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:20.517897 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:20.517897 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:20.517897 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:20.517897 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:20.517897 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:20.517897 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:20.517897 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:20.517897 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:20.517897 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:20.517897 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:20.517897 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:34:20.517897 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:34:20.517897 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:34:20.517897 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:34:20.518848 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:34:20.518848 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:34:20.518848 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:34:20.518848 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:34:20.518848 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:20.518848 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:20.518848 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:20.518848 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:20.518848 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:20.518848 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:20.518848 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:20.518848 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:20.518848 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:20.518848 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:20.518848 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:20.518848 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:20.519774 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:20.519774 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:20.519774 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:20.519774 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:20.519774 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:34:20.519774 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:34:20.519774 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:34:20.519774 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:34:20.519774 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:34:20.519774 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:34:20.519774 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 10:34:20.519774 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:20.519774 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 389/780] 10.222.0.15 () {56 vars in 1096 bytes} [Sun Sep 11 10:34:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:20.520557 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1249358e-a2d2-4fa1-946a-f06bdc3fd09a None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:20.523513 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c458bf6-1d6b-4c12-9053-6d7c2fba13ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:20.524591 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c458bf6-1d6b-4c12-9053-6d7c2fba13ce None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:20.525126 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c458bf6-1d6b-4c12-9053-6d7c2fba13ce None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:20.525726 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:20.526808 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c458bf6-1d6b-4c12-9053-6d7c2fba13ce None admin] PATH_INFO: `/v3/services` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:20.527823 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c458bf6-1d6b-4c12-9053-6d7c2fba13ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:20.529428 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c458bf6-1d6b-4c12-9053-6d7c2fba13ce None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:20.539479 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 392/781] 10.222.0.15 () {56 vars in 1087 bytes} [Sun Sep 11 10:34:20 2022] GET /identity/v3/services => generated 772 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:20.539479 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c458bf6-1d6b-4c12-9053-6d7c2fba13ce None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:20.668890 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1249358e-a2d2-4fa1-946a-f06bdc3fd09a None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:20.740067 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3b89e7-ebf7-4256-b516-1784a774706f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:20.742264 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3b89e7-ebf7-4256-b516-1784a774706f None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:20.742984 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3b89e7-ebf7-4256-b516-1784a774706f None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:20.743498 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3b89e7-ebf7-4256-b516-1784a774706f None admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:20.744491 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3b89e7-ebf7-4256-b516-1784a774706f None admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:20.744941 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3b89e7-ebf7-4256-b516-1784a774706f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:20.746845 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3b89e7-ebf7-4256-b516-1784a774706f None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:20.755625 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c458bf6-1d6b-4c12-9053-6d7c2fba13ce None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:20.761718 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 390/782] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:34:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:20.762751 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3b89e7-ebf7-4256-b516-1784a774706f None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:20.780017 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:20.811298 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a5d0f3-21eb-442f-84df-598d31741976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:20.813373 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a5d0f3-21eb-442f-84df-598d31741976 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:20.813653 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a5d0f3-21eb-442f-84df-598d31741976 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:20.813920 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a5d0f3-21eb-442f-84df-598d31741976 None admin] PATH_INFO: `/v3/roles/service` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:20.841520 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a5d0f3-21eb-442f-84df-598d31741976 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:34:20.850101 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-14a5d0f3-21eb-442f-84df-598d31741976 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 10:34:20.850101 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:20.850101 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:20.850101 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:20.850101 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:20.850101 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:20.850101 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:20.850101 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:20.850101 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:20.850101 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:20.850101 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:20.850101 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:20.850101 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:34:20.850101 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:34:20.850101 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:34:20.850101 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:34:20.851072 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:34:20.851072 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:34:20.851072 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:34:20.851072 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:34:20.851072 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:20.851072 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:20.851072 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:20.851072 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:20.851072 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:20.851072 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:20.851072 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:20.851072 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:20.851072 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:20.851072 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:20.851072 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:20.851072 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:20.852006 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:20.852006 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:20.852006 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:20.852006 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:20.852006 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:34:20.852006 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:34:20.852006 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:34:20.852006 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:34:20.852006 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:34:20.852006 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:34:20.852006 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 10:34:20.852006 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:20.852006 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 393/783] 10.222.0.15 () {56 vars in 1102 bytes} [Sun Sep 11 10:34:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:20.852969 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a5d0f3-21eb-442f-84df-598d31741976 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:20.861844 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:20.913739 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3b89e7-ebf7-4256-b516-1784a774706f None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:20.958065 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64da8ab-6b48-4166-8d5a-31f5492823d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:20.958984 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64da8ab-6b48-4166-8d5a-31f5492823d2 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:20.959409 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64da8ab-6b48-4166-8d5a-31f5492823d2 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:20.959840 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64da8ab-6b48-4166-8d5a-31f5492823d2 None admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:20.960646 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64da8ab-6b48-4166-8d5a-31f5492823d2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:20.961257 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64da8ab-6b48-4166-8d5a-31f5492823d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:20.962733 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64da8ab-6b48-4166-8d5a-31f5492823d2 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:20.976691 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 391/784] 10.222.0.15 () {56 vars in 1099 bytes} [Sun Sep 11 10:34:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:20.978249 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64da8ab-6b48-4166-8d5a-31f5492823d2 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:21.079356 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a5d0f3-21eb-442f-84df-598d31741976 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:21.117671 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64da8ab-6b48-4166-8d5a-31f5492823d2 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:21.155962 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:21.169294 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8090b24b-0288-429c-b100-8daf069cc10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:21.169954 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8090b24b-0288-429c-b100-8daf069cc10a None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:21.170298 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8090b24b-0288-429c-b100-8daf069cc10a None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:21.170575 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8090b24b-0288-429c-b100-8daf069cc10a None admin] PATH_INFO: `/v3/projects/service` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:21.176103 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e00121e-dbbd-4236-8e75-60fd1fc69da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:21.176720 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e00121e-dbbd-4236-8e75-60fd1fc69da8 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:21.177006 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e00121e-dbbd-4236-8e75-60fd1fc69da8 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:21.177277 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e00121e-dbbd-4236-8e75-60fd1fc69da8 None admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:21.177943 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e00121e-dbbd-4236-8e75-60fd1fc69da8 None admin] RBAC: Adding query filter params (name=service) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:21.182818 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e00121e-dbbd-4236-8e75-60fd1fc69da8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:21.185613 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8090b24b-0288-429c-b100-8daf069cc10a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:21.187082 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8090b24b-0288-429c-b100-8daf069cc10a None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:21.187650 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e00121e-dbbd-4236-8e75-60fd1fc69da8 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:21.206137 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 392/785] 10.222.0.15 () {56 vars in 1103 bytes} [Sun Sep 11 10:34:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:21.207147 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e00121e-dbbd-4236-8e75-60fd1fc69da8 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:21.218069 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-8090b24b-0288-429c-b100-8daf069cc10a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:34:21.218069 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:21.218069 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:21.218069 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:21.218069 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:21.218069 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:21.218069 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:21.218069 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:21.218069 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:21.218069 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:21.218069 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:21.218069 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:21.218069 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 10:34:21.218069 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 10:34:21.218069 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 10:34:21.219040 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 10:34:21.219040 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:21.219040 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:21.219040 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:21.219040 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:21.219040 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:21.219040 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:21.219040 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:21.219040 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:21.219040 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:21.219040 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:21.219040 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:21.219040 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:21.219040 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:21.219040 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:21.219040 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:21.219040 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:21.220052 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 10:34:21.220052 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 10:34:21.220052 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 10:34:21.220052 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 10:34:21.220052 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 10:34:21.220052 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 10:34:21.220052 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:34:21.220052 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:21.222770 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 394/786] 10.222.0.15 () {56 vars in 1111 bytes} [Sun Sep 11 10:34:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:21.226334 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8090b24b-0288-429c-b100-8daf069cc10a None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:21.300342 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:21.378520 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e00121e-dbbd-4236-8e75-60fd1fc69da8 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:21.399645 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8090b24b-0288-429c-b100-8daf069cc10a None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:21.417397 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f740a9f8-bf00-4ad8-84ed-cd84166b48ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:21.418036 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f740a9f8-bf00-4ad8-84ed-cd84166b48ca None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:21.418313 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f740a9f8-bf00-4ad8-84ed-cd84166b48ca None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:21.418581 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f740a9f8-bf00-4ad8-84ed-cd84166b48ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:21.430757 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f740a9f8-bf00-4ad8-84ed-cd84166b48ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:21.432269 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f740a9f8-bf00-4ad8-84ed-cd84166b48ca None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:21.443364 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba9581a-9200-4017-bbbf-363898fb70e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:21.444713 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba9581a-9200-4017-bbbf-363898fb70e5 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:21.445194 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba9581a-9200-4017-bbbf-363898fb70e5 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:21.445664 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba9581a-9200-4017-bbbf-363898fb70e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:21.448064 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-f740a9f8-bf00-4ad8-84ed-cd84166b48ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:21.448064 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:21.448064 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:21.448064 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:21.448064 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:21.448064 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:21.448064 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:21.448064 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:21.448064 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:21.448064 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:21.448064 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:21.448064 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:21.448064 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:34:21.448064 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:34:21.448064 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:34:21.448064 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:34:21.448995 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:21.448995 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:21.448995 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:21.448995 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:21.448995 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:21.448995 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:21.448995 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:21.448995 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:21.448995 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:21.448995 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:21.448995 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:21.448995 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:21.448995 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:21.448995 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:21.448995 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:21.448995 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:21.448995 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:34:21.449971 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:34:21.449971 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:21.449971 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:21.449971 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 393/787] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:34:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:21.450699 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f740a9f8-bf00-4ad8-84ed-cd84166b48ca None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:21.466255 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba9581a-9200-4017-bbbf-363898fb70e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:21.468857 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba9581a-9200-4017-bbbf-363898fb70e5 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:21.486846 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-5ba9581a-9200-4017-bbbf-363898fb70e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:21.486846 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:21.486846 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:21.486846 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:21.486846 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:21.486846 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:21.486846 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:21.486846 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:21.486846 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:21.486846 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:21.486846 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:21.486846 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:21.486846 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:34:21.486846 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:34:21.486846 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:34:21.486846 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:34:21.487873 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:21.487873 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:21.487873 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:21.487873 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:21.487873 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:21.487873 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:21.487873 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:21.487873 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:21.487873 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:21.487873 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:21.487873 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:21.487873 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:21.487873 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:21.487873 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:21.487873 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:21.487873 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:21.487873 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:34:21.488757 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:34:21.488757 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:21.488757 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:21.495942 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 395/788] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:34:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:21.497491 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f36deb7-bd64-4439-b828-ef7a6f65c589 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:21.497984 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f36deb7-bd64-4439-b828-ef7a6f65c589 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:21.498451 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f36deb7-bd64-4439-b828-ef7a6f65c589 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:21.503742 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 396/789] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:34:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:34:21.503742 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f36deb7-bd64-4439-b828-ef7a6f65c589 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:21.510402 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:21.586462 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f740a9f8-bf00-4ad8-84ed-cd84166b48ca None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:21.622129 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:21.628257 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d9d8a9-4779-4155-93fb-7b7658c214a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:21.628930 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d9d8a9-4779-4155-93fb-7b7658c214a8 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:21.629162 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d9d8a9-4779-4155-93fb-7b7658c214a8 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:21.629600 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d9d8a9-4779-4155-93fb-7b7658c214a8 None admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:21.630281 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d9d8a9-4779-4155-93fb-7b7658c214a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:21.630831 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d9d8a9-4779-4155-93fb-7b7658c214a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:21.632468 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d9d8a9-4779-4155-93fb-7b7658c214a8 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:21.644023 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f36deb7-bd64-4439-b828-ef7a6f65c589 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:21.647581 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 394/790] 10.222.0.15 () {56 vars in 1148 bytes} [Sun Sep 11 10:34:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:21.648911 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d9d8a9-4779-4155-93fb-7b7658c214a8 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:21.694858 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb81d6d-9132-4d37-936b-e33a44a95efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:21.695830 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb81d6d-9132-4d37-936b-e33a44a95efe None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:21.696209 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb81d6d-9132-4d37-936b-e33a44a95efe None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:21.696616 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb81d6d-9132-4d37-936b-e33a44a95efe None admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:21.697425 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb81d6d-9132-4d37-936b-e33a44a95efe None admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:21.697834 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb81d6d-9132-4d37-936b-e33a44a95efe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:21.699346 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb81d6d-9132-4d37-936b-e33a44a95efe None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:21.714368 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 397/791] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:34:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:21.720163 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6124576a-1224-4074-8bcb-bb6e44bcca13 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:21.720450 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6124576a-1224-4074-8bcb-bb6e44bcca13 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:21.720722 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6124576a-1224-4074-8bcb-bb6e44bcca13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:21.745599 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-6124576a-1224-4074-8bcb-bb6e44bcca13 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:34:21.747416 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-6124576a-1224-4074-8bcb-bb6e44bcca13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:34:21.773495 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d9d8a9-4779-4155-93fb-7b7658c214a8 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:21.775686 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:21.820803 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df2067e-8fcc-453e-a439-d2fe5ba78642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:21.821419 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df2067e-8fcc-453e-a439-d2fe5ba78642 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:21.821712 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df2067e-8fcc-453e-a439-d2fe5ba78642 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:21.821986 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df2067e-8fcc-453e-a439-d2fe5ba78642 None admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:21.822594 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df2067e-8fcc-453e-a439-d2fe5ba78642 None admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:21.822892 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df2067e-8fcc-453e-a439-d2fe5ba78642 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:21.824698 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df2067e-8fcc-453e-a439-d2fe5ba78642 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:21.833145 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 398/792] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:34:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 10:34:21.836195 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6124576a-1224-4074-8bcb-bb6e44bcca13 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:21.859505 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 395/793] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:34:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:21.859505 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df2067e-8fcc-453e-a439-d2fe5ba78642 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:21.931032 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:21.973720 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6124576a-1224-4074-8bcb-bb6e44bcca13 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:22.037894 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abae572-fa75-4cc7-9567-ea8289b3e496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:22.038708 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abae572-fa75-4cc7-9567-ea8289b3e496 None admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:22.039082 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abae572-fa75-4cc7-9567-ea8289b3e496 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:22.039464 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abae572-fa75-4cc7-9567-ea8289b3e496 None admin] PATH_INFO: `/v3/projects/5e1f845f99f04202b83621cb65e0adc2/users/2bd5356829b6454993bab9bad2f79fed/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:22.042068 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abae572-fa75-4cc7-9567-ea8289b3e496 None admin] RBAC: Authorizing `identity:create_grant(project_id=5e1f845f99f04202b83621cb65e0adc2, user_id=2bd5356829b6454993bab9bad2f79fed, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:22.044050 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abae572-fa75-4cc7-9567-ea8289b3e496 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:22.052572 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df2067e-8fcc-453e-a439-d2fe5ba78642 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:22.083513 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 399/794] 10.222.0.15 () {58 vars in 1439 bytes} [Sun Sep 11 10:34:21 2022] PUT /identity/v3/projects/5e1f845f99f04202b83621cb65e0adc2/users/2bd5356829b6454993bab9bad2f79fed/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:34:22.085407 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abae572-fa75-4cc7-9567-ea8289b3e496 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:22.119035 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ee7889-e4eb-4bc1-9f4b-0858a435d571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:22.121438 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee7889-e4eb-4bc1-9f4b-0858a435d571 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:22.122004 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee7889-e4eb-4bc1-9f4b-0858a435d571 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:22.122629 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee7889-e4eb-4bc1-9f4b-0858a435d571 None admin] PATH_INFO: `/v3/users/swift` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:22.143948 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ee7889-e4eb-4bc1-9f4b-0858a435d571 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:22.145681 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ee7889-e4eb-4bc1-9f4b-0858a435d571 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:22.160862 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-98ee7889-e4eb-4bc1-9f4b-0858a435d571 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 11 10:34:22.160862 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:22.160862 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:22.160862 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:22.160862 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:22.160862 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:22.160862 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:22.160862 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:22.160862 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:22.160862 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:22.160862 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:22.160862 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:22.160862 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 10:34:22.160862 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 10:34:22.160862 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 10:34:22.160862 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 10:34:22.162111 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:22.162111 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:22.162111 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 10:34:22.162111 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:34:22.162111 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 10:34:22.162111 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:34:22.162111 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:22.162111 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:22.162111 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:22.162111 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:22.162111 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:22.162111 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:22.162111 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:22.162111 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:22.162111 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:22.162111 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:22.162111 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:22.162111 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:22.163465 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:22.163465 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:22.163465 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 10:34:22.163465 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 10:34:22.163465 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 10:34:22.163465 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 10:34:22.163465 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 10:34:22.163465 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 10:34:22.163465 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 11 10:34:22.163465 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:22.173438 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 396/795] 10.222.0.15 () {56 vars in 1096 bytes} [Sun Sep 11 10:34:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:22.174357 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ee7889-e4eb-4bc1-9f4b-0858a435d571 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:22.241674 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:22.370990 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ee7889-e4eb-4bc1-9f4b-0858a435d571 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:22.403754 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abae572-fa75-4cc7-9567-ea8289b3e496 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:22.408645 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6e2de5-def8-41b5-8e52-319aeb2b119f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:22.409540 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e2de5-def8-41b5-8e52-319aeb2b119f None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:22.410043 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e2de5-def8-41b5-8e52-319aeb2b119f None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:22.410521 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e2de5-def8-41b5-8e52-319aeb2b119f None admin] PATH_INFO: `/v3/services/network` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:22.411348 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6e2de5-def8-41b5-8e52-319aeb2b119f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:22.413515 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6e2de5-def8-41b5-8e52-319aeb2b119f None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:22.423826 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-ff6e2de5-def8-41b5-8e52-319aeb2b119f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 11 10:34:22.423826 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:22.423826 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 10:34:22.423826 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 10:34:22.423826 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 10:34:22.423826 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 10:34:22.423826 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 10:34:22.423826 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 10:34:22.423826 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 11 10:34:22.423826 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:22.423826 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 10:34:22.423826 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:22.423826 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:22.423826 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:22.423826 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:22.423826 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:22.423826 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:22.424879 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:22.424879 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:22.424879 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:22.424879 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:22.424879 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:22.424879 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:22.424879 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 10:34:22.424879 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 10:34:22.424879 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 10:34:22.424879 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 10:34:22.424879 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:22.424879 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:22.424879 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:22.424879 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:22.424879 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:22.424879 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:22.425769 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:22.425769 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:22.425769 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:22.425769 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:22.425769 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:22.425769 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:22.425769 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:22.425769 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:22.425769 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:22.425769 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:22.425769 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 10:34:22.425769 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 10:34:22.425769 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 11 10:34:22.425769 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:22.428732 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 397/796] 10.222.0.15 () {56 vars in 1111 bytes} [Sun Sep 11 10:34:22 2022] GET /identity/v3/services/network => generated 88 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:22.429739 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6e2de5-def8-41b5-8e52-319aeb2b119f None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:22.444213 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fad329-f09d-4b54-9d92-e28646dcc83b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:22.444857 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fad329-f09d-4b54-9d92-e28646dcc83b None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:22.445135 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fad329-f09d-4b54-9d92-e28646dcc83b None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:22.445409 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fad329-f09d-4b54-9d92-e28646dcc83b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:22.449345 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:22.459120 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fad329-f09d-4b54-9d92-e28646dcc83b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:22.462084 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fad329-f09d-4b54-9d92-e28646dcc83b None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:22.488519 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-98fad329-f09d-4b54-9d92-e28646dcc83b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 10:34:22.488519 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:22.488519 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:22.488519 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:22.488519 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:22.488519 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:22.488519 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:22.488519 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:22.488519 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:22.488519 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:22.488519 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:22.488519 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:22.488519 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 10:34:22.488519 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 10:34:22.488519 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 10:34:22.488519 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 10:34:22.489531 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:22.489531 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:22.489531 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 10:34:22.489531 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:34:22.489531 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 10:34:22.489531 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:34:22.489531 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:22.489531 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:22.489531 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:22.489531 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:22.489531 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:22.489531 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:22.489531 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:22.489531 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:22.489531 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:22.489531 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:22.489531 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:22.489531 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:22.490492 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:22.490492 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:22.490492 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 10:34:22.490492 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 10:34:22.490492 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 10:34:22.490492 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 10:34:22.490492 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 10:34:22.490492 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 10:34:22.490492 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 10:34:22.490492 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:22.498891 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 400/797] 10.222.0.15 () {56 vars in 1117 bytes} [Sun Sep 11 10:34:22 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 410 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:22.500836 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fad329-f09d-4b54-9d92-e28646dcc83b None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:22.609445 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:22.642506 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6e2de5-def8-41b5-8e52-319aeb2b119f None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:22.713831 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fad329-f09d-4b54-9d92-e28646dcc83b None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:22.714859 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aedf9d9-6b64-4b67-887f-1222c6c97ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:22.717315 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aedf9d9-6b64-4b67-887f-1222c6c97ca7 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:22.717813 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aedf9d9-6b64-4b67-887f-1222c6c97ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:22.718284 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aedf9d9-6b64-4b67-887f-1222c6c97ca7 None admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:22.719208 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aedf9d9-6b64-4b67-887f-1222c6c97ca7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:22.719794 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aedf9d9-6b64-4b67-887f-1222c6c97ca7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:22.726208 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aedf9d9-6b64-4b67-887f-1222c6c97ca7 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:22.766436 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 398/798] 10.222.0.15 () {56 vars in 1135 bytes} [Sun Sep 11 10:34:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:22.767627 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aedf9d9-6b64-4b67-887f-1222c6c97ca7 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:22.789530 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc21b51-7a8f-426c-b726-cf166fd8de3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:22.790727 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc21b51-7a8f-426c-b726-cf166fd8de3a None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:22.792338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc21b51-7a8f-426c-b726-cf166fd8de3a None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:22.793015 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc21b51-7a8f-426c-b726-cf166fd8de3a None admin] PATH_INFO: `/v3/services` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:22.794537 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc21b51-7a8f-426c-b726-cf166fd8de3a None admin] RBAC: Adding query filter params (name=network) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:22.796539 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc21b51-7a8f-426c-b726-cf166fd8de3a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:22.798129 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc21b51-7a8f-426c-b726-cf166fd8de3a None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:22.815112 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 401/799] 10.222.0.15 () {56 vars in 1112 bytes} [Sun Sep 11 10:34:22 2022] GET /identity/v3/services?name=network => generated 124 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:22.816320 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc21b51-7a8f-426c-b726-cf166fd8de3a None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:22.952117 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:22.958193 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc21b51-7a8f-426c-b726-cf166fd8de3a None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:23.002122 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcba8f79-9d5e-4f50-abe8-bec243646b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:23.003120 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcba8f79-9d5e-4f50-abe8-bec243646b14 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:23.003652 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcba8f79-9d5e-4f50-abe8-bec243646b14 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:23.004137 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcba8f79-9d5e-4f50-abe8-bec243646b14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:23.018840 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcba8f79-9d5e-4f50-abe8-bec243646b14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:23.020812 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcba8f79-9d5e-4f50-abe8-bec243646b14 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:23.028077 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aedf9d9-6b64-4b67-887f-1222c6c97ca7 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:23.038178 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-dcba8f79-9d5e-4f50-abe8-bec243646b14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:23.038178 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:23.038178 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:23.038178 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:23.038178 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:23.038178 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:23.038178 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:23.038178 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:23.038178 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:23.038178 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:23.038178 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:23.038178 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:23.038178 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:34:23.038178 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:34:23.038178 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:34:23.038178 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:34:23.039154 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:23.039154 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:23.039154 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:23.039154 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:23.039154 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:23.039154 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:23.039154 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:23.039154 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:23.039154 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:23.039154 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:23.039154 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:23.039154 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:23.039154 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:23.039154 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:23.039154 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:23.039154 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:23.039154 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:34:23.040076 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:34:23.040076 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:23.040076 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:23.042261 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 402/800] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:34:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:23.043366 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcba8f79-9d5e-4f50-abe8-bec243646b14 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:23.069711 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff393356-1aca-489e-b84a-faefa57de939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:23.070383 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff393356-1aca-489e-b84a-faefa57de939 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:23.070670 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff393356-1aca-489e-b84a-faefa57de939 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:23.071119 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff393356-1aca-489e-b84a-faefa57de939 None admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:23.072067 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff393356-1aca-489e-b84a-faefa57de939 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:23.072311 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff393356-1aca-489e-b84a-faefa57de939 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:23.074549 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff393356-1aca-489e-b84a-faefa57de939 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:23.093578 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:23.103212 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 399/801] 10.222.0.15 () {56 vars in 1149 bytes} [Sun Sep 11 10:34:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:23.103212 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433fd117-0bf9-49ad-8cba-07062380619d None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:23.103212 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433fd117-0bf9-49ad-8cba-07062380619d None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:23.103212 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433fd117-0bf9-49ad-8cba-07062380619d None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:23.107667 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 400/802] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:34:23 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:34:23.108630 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433fd117-0bf9-49ad-8cba-07062380619d None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:23.205317 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcba8f79-9d5e-4f50-abe8-bec243646b14 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:23.228318 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:23.229737 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433fd117-0bf9-49ad-8cba-07062380619d None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:23.267447 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac67f5ef-9715-4efa-ac87-72745ce5e571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:23.268263 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67f5ef-9715-4efa-ac87-72745ce5e571 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:23.268693 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67f5ef-9715-4efa-ac87-72745ce5e571 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:23.269011 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67f5ef-9715-4efa-ac87-72745ce5e571 None admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:23.269624 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac67f5ef-9715-4efa-ac87-72745ce5e571 None admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:23.270027 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac67f5ef-9715-4efa-ac87-72745ce5e571 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:23.271407 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac67f5ef-9715-4efa-ac87-72745ce5e571 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:23.280596 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d062829-b3f9-43c6-9e94-1830a9c023e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:23.282002 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d062829-b3f9-43c6-9e94-1830a9c023e3 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:23.282387 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d062829-b3f9-43c6-9e94-1830a9c023e3 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:23.282760 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d062829-b3f9-43c6-9e94-1830a9c023e3 None admin] PATH_INFO: `/v3/services` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:23.285008 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d062829-b3f9-43c6-9e94-1830a9c023e3 None admin] RBAC: Adding query filter params (type=network) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:23.285518 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 401/803] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:34:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:23.285679 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d062829-b3f9-43c6-9e94-1830a9c023e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:23.286766 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac67f5ef-9715-4efa-ac87-72745ce5e571 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:23.287063 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d062829-b3f9-43c6-9e94-1830a9c023e3 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:23.314253 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 403/804] 10.222.0.15 () {56 vars in 1112 bytes} [Sun Sep 11 10:34:23 2022] GET /identity/v3/services?type=network => generated 350 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:23.315358 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d41b0d1-46fd-40de-8be8-de6d3a78e182 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:23.315744 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d41b0d1-46fd-40de-8be8-de6d3a78e182 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:23.316091 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d41b0d1-46fd-40de-8be8-de6d3a78e182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:23.334229 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-5d41b0d1-46fd-40de-8be8-de6d3a78e182 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:34:23.336055 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-5d41b0d1-46fd-40de-8be8-de6d3a78e182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:34:23.383993 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:23.439601 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 404/805] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:34:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 10:34:23.440554 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d41b0d1-46fd-40de-8be8-de6d3a78e182 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:23.475615 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac67f5ef-9715-4efa-ac87-72745ce5e571 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:23.606914 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fda83d-9610-4d0c-b066-e2af7fbe9259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:23.614157 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fda83d-9610-4d0c-b066-e2af7fbe9259 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:23.615876 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fda83d-9610-4d0c-b066-e2af7fbe9259 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:23.616409 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fda83d-9610-4d0c-b066-e2af7fbe9259 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:23.654226 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fda83d-9610-4d0c-b066-e2af7fbe9259 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:23.659573 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fda83d-9610-4d0c-b066-e2af7fbe9259 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:23.729409 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-09fda83d-9610-4d0c-b066-e2af7fbe9259 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:23.729409 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:23.729409 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:23.729409 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:23.729409 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:23.729409 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:23.729409 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:23.729409 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:23.729409 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:23.729409 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:23.729409 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:23.729409 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:23.729409 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:34:23.729409 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:34:23.729409 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:34:23.729409 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:34:23.730529 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:23.730529 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:23.730529 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:23.730529 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:23.730529 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:23.730529 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:23.730529 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:23.730529 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:23.730529 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:23.730529 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:23.730529 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:23.730529 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:23.730529 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:23.730529 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:23.730529 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:23.730529 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:23.730529 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:34:23.731842 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:34:23.731842 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:23.731842 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:23.731842 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 402/806] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:34:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 445 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:23.731842 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fda83d-9610-4d0c-b066-e2af7fbe9259 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:23.754340 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d41b0d1-46fd-40de-8be8-de6d3a78e182 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:23.762481 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:23.806765 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d477218e-72ae-447d-af0b-dde58aed1dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:23.807393 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d477218e-72ae-447d-af0b-dde58aed1dc3 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:23.807817 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d477218e-72ae-447d-af0b-dde58aed1dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:23.808104 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d477218e-72ae-447d-af0b-dde58aed1dc3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:23.819374 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d477218e-72ae-447d-af0b-dde58aed1dc3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:23.821461 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d477218e-72ae-447d-af0b-dde58aed1dc3 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:23.848402 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-d477218e-72ae-447d-af0b-dde58aed1dc3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:34:23.848402 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:23.848402 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:23.848402 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:23.848402 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:23.848402 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:23.848402 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:23.848402 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:23.848402 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:23.848402 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:23.848402 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:23.848402 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:23.848402 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 10:34:23.848402 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 10:34:23.848402 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 10:34:23.849544 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 10:34:23.849544 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:23.849544 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:23.849544 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:23.849544 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:23.849544 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:23.849544 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:23.849544 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:23.849544 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:23.849544 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:23.849544 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:23.849544 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:23.849544 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:23.849544 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:23.849544 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:23.849544 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:23.849544 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:23.850762 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 10:34:23.850762 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 10:34:23.850762 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 10:34:23.850762 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 10:34:23.850762 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 10:34:23.850762 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 10:34:23.850762 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:34:23.850762 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:23.850762 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 405/807] 10.222.0.15 () {56 vars in 1111 bytes} [Sun Sep 11 10:34:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 410 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:23.851454 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d477218e-72ae-447d-af0b-dde58aed1dc3 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:23.883089 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fda83d-9610-4d0c-b066-e2af7fbe9259 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:23.900887 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:23.924956 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3392b97-9d33-4e36-9b99-1dab01491887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:23.925568 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3392b97-9d33-4e36-9b99-1dab01491887 None admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:23.925851 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3392b97-9d33-4e36-9b99-1dab01491887 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:23.926127 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3392b97-9d33-4e36-9b99-1dab01491887 None admin] PATH_INFO: `/v3/endpoints` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:23.926786 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3392b97-9d33-4e36-9b99-1dab01491887 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:23.928219 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3392b97-9d33-4e36-9b99-1dab01491887 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:23.963453 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 403/808] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:34:23 2022] POST /identity/v3/endpoints => generated 337 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:34:23.963453 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3392b97-9d33-4e36-9b99-1dab01491887 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:23.977729 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d477218e-72ae-447d-af0b-dde58aed1dc3 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:24.031841 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e3ecc3-2052-48fe-94c9-bcc8dfe2e0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:24.032525 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e3ecc3-2052-48fe-94c9-bcc8dfe2e0d1 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:24.032981 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e3ecc3-2052-48fe-94c9-bcc8dfe2e0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:24.033261 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e3ecc3-2052-48fe-94c9-bcc8dfe2e0d1 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:24.033856 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e3ecc3-2052-48fe-94c9-bcc8dfe2e0d1 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:24.035249 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e3ecc3-2052-48fe-94c9-bcc8dfe2e0d1 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:24.046494 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-c5e3ecc3-2052-48fe-94c9-bcc8dfe2e0d1 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 11 10:34:24.046494 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:24.046494 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 10:34:24.046494 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 10:34:24.046494 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 10:34:24.046494 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 10:34:24.046494 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 10:34:24.046494 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 10:34:24.046494 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 11 10:34:24.046494 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:24.046494 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 10:34:24.046494 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:24.046494 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:24.046494 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:24.046494 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:24.046494 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:24.046494 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:24.047954 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:24.047954 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:24.047954 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:24.047954 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:24.047954 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:24.047954 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:24.047954 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 10:34:24.047954 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 10:34:24.047954 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 10:34:24.047954 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 10:34:24.047954 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:24.047954 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:24.047954 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:24.047954 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:24.047954 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:24.047954 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:24.048835 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:24.048835 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:24.048835 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:24.048835 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:24.048835 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:24.048835 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:24.048835 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:24.048835 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:24.048835 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:24.048835 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:24.048835 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 10:34:24.048835 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 10:34:24.048835 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 11 10:34:24.048835 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:24.048835 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 406/809] 10.222.0.15 () {56 vars in 1129 bytes} [Sun Sep 11 10:34:23 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:24.049647 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e3ecc3-2052-48fe-94c9-bcc8dfe2e0d1 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:24.089178 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:24.123824 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3392b97-9d33-4e36-9b99-1dab01491887 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:24.174210 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16854926-507f-4a40-a870-f6a9dbc6361e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:24.175074 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16854926-507f-4a40-a870-f6a9dbc6361e None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:24.175354 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16854926-507f-4a40-a870-f6a9dbc6361e None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:24.175660 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16854926-507f-4a40-a870-f6a9dbc6361e None admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:24.176310 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16854926-507f-4a40-a870-f6a9dbc6361e None admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:24.176616 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16854926-507f-4a40-a870-f6a9dbc6361e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:24.177982 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16854926-507f-4a40-a870-f6a9dbc6361e None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:24.185518 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e3ecc3-2052-48fe-94c9-bcc8dfe2e0d1 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:24.195023 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 404/810] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:34:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:24.195877 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16854926-507f-4a40-a870-f6a9dbc6361e None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:24.232391 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:24.238972 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae66762c-2c7b-4939-8010-ebe125319beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:24.239704 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae66762c-2c7b-4939-8010-ebe125319beb None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:24.239981 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae66762c-2c7b-4939-8010-ebe125319beb None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:24.240250 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae66762c-2c7b-4939-8010-ebe125319beb None admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:24.241598 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae66762c-2c7b-4939-8010-ebe125319beb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:24.241902 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae66762c-2c7b-4939-8010-ebe125319beb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:24.243303 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae66762c-2c7b-4939-8010-ebe125319beb None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:24.256581 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 407/811] 10.222.0.15 () {56 vars in 1148 bytes} [Sun Sep 11 10:34:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:24.257679 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae66762c-2c7b-4939-8010-ebe125319beb None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:24.324826 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16854926-507f-4a40-a870-f6a9dbc6361e None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:24.360549 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:24.364813 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f4a542-cf1c-4b2f-ad1c-be050b3bfd77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:24.365403 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f4a542-cf1c-4b2f-ad1c-be050b3bfd77 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:24.366251 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f4a542-cf1c-4b2f-ad1c-be050b3bfd77 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:24.366588 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f4a542-cf1c-4b2f-ad1c-be050b3bfd77 None admin] PATH_INFO: `/v3/services` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:24.367310 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f4a542-cf1c-4b2f-ad1c-be050b3bfd77 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:24.367752 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f4a542-cf1c-4b2f-ad1c-be050b3bfd77 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:24.369215 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f4a542-cf1c-4b2f-ad1c-be050b3bfd77 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:24.374525 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 405/812] 10.222.0.15 () {56 vars in 1124 bytes} [Sun Sep 11 10:34:24 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:24.375343 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f4a542-cf1c-4b2f-ad1c-be050b3bfd77 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:24.386250 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae66762c-2c7b-4939-8010-ebe125319beb None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:24.429070 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac816c2d-0d45-454f-a61d-848c2fcf561a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:24.429859 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac816c2d-0d45-454f-a61d-848c2fcf561a None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:24.430206 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac816c2d-0d45-454f-a61d-848c2fcf561a None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:24.430919 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac816c2d-0d45-454f-a61d-848c2fcf561a None admin] PATH_INFO: `/v3/projects/service` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:24.438645 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac816c2d-0d45-454f-a61d-848c2fcf561a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:24.440189 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac816c2d-0d45-454f-a61d-848c2fcf561a None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:24.452615 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-ac816c2d-0d45-454f-a61d-848c2fcf561a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:34:24.452615 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:24.452615 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:24.452615 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:24.452615 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:24.452615 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:24.452615 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:24.452615 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:24.452615 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:24.452615 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:24.452615 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:24.452615 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:24.452615 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 10:34:24.452615 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 10:34:24.452615 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 10:34:24.453599 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 10:34:24.453599 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:24.453599 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:24.453599 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:24.453599 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:24.453599 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:24.453599 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:24.453599 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:24.453599 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:24.453599 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:24.453599 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:24.453599 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:24.453599 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:24.453599 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:24.453599 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:24.453599 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:24.453599 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:24.460193 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 10:34:24.460193 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 10:34:24.460193 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 10:34:24.460193 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 10:34:24.460193 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 10:34:24.460193 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 10:34:24.460193 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:34:24.460193 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:24.460193 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 408/813] 10.222.0.15 () {56 vars in 1111 bytes} [Sun Sep 11 10:34:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:24.460193 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac816c2d-0d45-454f-a61d-848c2fcf561a None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:24.475256 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:24.485028 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f4a542-cf1c-4b2f-ad1c-be050b3bfd77 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:24.519003 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39303a62-f5f5-4b99-9669-60df881a7d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:24.519713 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39303a62-f5f5-4b99-9669-60df881a7d7d None admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:24.519995 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39303a62-f5f5-4b99-9669-60df881a7d7d None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:24.520267 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39303a62-f5f5-4b99-9669-60df881a7d7d None admin] PATH_INFO: `/v3/projects/5e1f845f99f04202b83621cb65e0adc2/users/cc7ca566e66a45b29e3d02f47bf84981/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:24.522309 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39303a62-f5f5-4b99-9669-60df881a7d7d None admin] RBAC: Authorizing `identity:create_grant(project_id=5e1f845f99f04202b83621cb65e0adc2, user_id=cc7ca566e66a45b29e3d02f47bf84981, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:24.523790 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39303a62-f5f5-4b99-9669-60df881a7d7d None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:24.536003 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 406/814] 10.222.0.15 () {58 vars in 1439 bytes} [Sun Sep 11 10:34:24 2022] PUT /identity/v3/projects/5e1f845f99f04202b83621cb65e0adc2/users/cc7ca566e66a45b29e3d02f47bf84981/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:34:24.536795 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39303a62-f5f5-4b99-9669-60df881a7d7d None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:24.566007 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac816c2d-0d45-454f-a61d-848c2fcf561a None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:24.604934 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6141fb1-1d73-45bf-84af-c1cd80d01ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:24.605917 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6141fb1-1d73-45bf-84af-c1cd80d01ded None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:24.606449 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6141fb1-1d73-45bf-84af-c1cd80d01ded None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:24.606801 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6141fb1-1d73-45bf-84af-c1cd80d01ded None admin] PATH_INFO: `/v3/services` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:24.607455 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6141fb1-1d73-45bf-84af-c1cd80d01ded None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:24.607758 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6141fb1-1d73-45bf-84af-c1cd80d01ded None admin] RBAC: Authorizing `identity:list_services()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:24.609191 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6141fb1-1d73-45bf-84af-c1cd80d01ded None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:24.614630 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 409/815] 10.222.0.15 () {56 vars in 1124 bytes} [Sun Sep 11 10:34:24 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:24.616558 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755cd5bb-ce1e-4660-bc98-1193397787b9 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:24.616844 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755cd5bb-ce1e-4660-bc98-1193397787b9 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:24.617101 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755cd5bb-ce1e-4660-bc98-1193397787b9 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:24.618609 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 410/816] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:34:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:34:24.623740 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755cd5bb-ce1e-4660-bc98-1193397787b9 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:24.637868 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:24.669079 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39303a62-f5f5-4b99-9669-60df881a7d7d None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:24.708000 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0288cf55-4b54-401f-9f8c-dff9c7bc296c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:24.708961 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0288cf55-4b54-401f-9f8c-dff9c7bc296c None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:24.708961 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0288cf55-4b54-401f-9f8c-dff9c7bc296c None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:24.709168 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0288cf55-4b54-401f-9f8c-dff9c7bc296c None admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:24.709876 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0288cf55-4b54-401f-9f8c-dff9c7bc296c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:24.710134 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0288cf55-4b54-401f-9f8c-dff9c7bc296c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:24.712181 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0288cf55-4b54-401f-9f8c-dff9c7bc296c None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:24.726372 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 407/817] 10.222.0.15 () {56 vars in 1148 bytes} [Sun Sep 11 10:34:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:24.727966 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cfeec6-245c-4fc8-b22d-29ae1c1b2ced None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:24.728255 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cfeec6-245c-4fc8-b22d-29ae1c1b2ced None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:24.728603 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cfeec6-245c-4fc8-b22d-29ae1c1b2ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:24.745852 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755cd5bb-ce1e-4660-bc98-1193397787b9 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:24.749304 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-d0cfeec6-245c-4fc8-b22d-29ae1c1b2ced None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:34:24.750494 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-d0cfeec6-245c-4fc8-b22d-29ae1c1b2ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:34:24.759366 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:24.781884 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bece708b-aa20-4a00-9431-16b790bca698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:24.782495 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bece708b-aa20-4a00-9431-16b790bca698 None admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:24.782778 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bece708b-aa20-4a00-9431-16b790bca698 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:24.783047 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bece708b-aa20-4a00-9431-16b790bca698 None admin] PATH_INFO: `/v3/endpoints` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:24.783737 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bece708b-aa20-4a00-9431-16b790bca698 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:24.785094 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bece708b-aa20-4a00-9431-16b790bca698 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:24.808190 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 411/818] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:34:24 2022] POST /identity/v3/endpoints => generated 346 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:34:24.809405 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bece708b-aa20-4a00-9431-16b790bca698 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:24.843217 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 408/819] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:34:24 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 10:34:24.923398 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:24.957086 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bece708b-aa20-4a00-9431-16b790bca698 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:24.999255 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455c4020-a111-465d-98e1-975f240c1027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:24.999486 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c4020-a111-465d-98e1-975f240c1027 None admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:24.999879 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c4020-a111-465d-98e1-975f240c1027 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:25.000246 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c4020-a111-465d-98e1-975f240c1027 None admin] PATH_INFO: `/v3/projects/5e1f845f99f04202b83621cb65e0adc2/users/bb44e7aef0c7436d93a7233307b5f095/roles/3cb7ebbc707d453586b3d0f4ab3cf103` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:25.020886 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455c4020-a111-465d-98e1-975f240c1027 None admin] RBAC: Authorizing `identity:create_grant(project_id=5e1f845f99f04202b83621cb65e0adc2, user_id=bb44e7aef0c7436d93a7233307b5f095, role_id=3cb7ebbc707d453586b3d0f4ab3cf103)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:25.022841 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455c4020-a111-465d-98e1-975f240c1027 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:25.040529 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 412/820] 10.222.0.15 () {58 vars in 1439 bytes} [Sun Sep 11 10:34:24 2022] PUT /identity/v3/projects/5e1f845f99f04202b83621cb65e0adc2/users/bb44e7aef0c7436d93a7233307b5f095/roles/3cb7ebbc707d453586b3d0f4ab3cf103 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:34:25.146359 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cfeec6-245c-4fc8-b22d-29ae1c1b2ced None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:25.290292 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cfeec6-245c-4fc8-b22d-29ae1c1b2ced None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:25.337059 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e311b4e-5d14-4cee-ac91-fc67a47774bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:25.337776 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e311b4e-5d14-4cee-ac91-fc67a47774bb None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:25.338063 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e311b4e-5d14-4cee-ac91-fc67a47774bb None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:25.338342 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e311b4e-5d14-4cee-ac91-fc67a47774bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:25.346028 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:25.350618 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e311b4e-5d14-4cee-ac91-fc67a47774bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:34:25.356320 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-3e311b4e-5d14-4cee-ac91-fc67a47774bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 10:34:25.356320 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:25.356320 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:25.356320 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:25.356320 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:25.356320 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:25.356320 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:25.356320 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:25.356320 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:25.356320 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:25.356320 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:25.356320 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:25.356320 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:34:25.356320 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:34:25.356320 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:34:25.356320 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:34:25.357465 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:34:25.357465 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:34:25.357465 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:34:25.357465 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:34:25.357465 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:25.357465 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:25.357465 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:25.357465 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:25.357465 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:25.357465 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:25.357465 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:25.357465 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:25.357465 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:25.357465 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:25.357465 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:25.357465 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:25.358586 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:25.358586 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:25.358586 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:25.358586 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:25.358586 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:34:25.358586 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:34:25.358586 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:34:25.358586 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:34:25.358586 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:34:25.358586 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:34:25.358586 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 10:34:25.358586 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:25.362496 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 409/821] 10.222.0.15 () {56 vars in 1096 bytes} [Sun Sep 11 10:34:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:25.364709 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455c4020-a111-465d-98e1-975f240c1027 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:25.503454 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455c4020-a111-465d-98e1-975f240c1027 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:25.535828 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:25.542174 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf318382-99c9-40e7-96a7-dae80e5488d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:25.542733 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf318382-99c9-40e7-96a7-dae80e5488d4 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:25.543009 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf318382-99c9-40e7-96a7-dae80e5488d4 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:25.543276 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf318382-99c9-40e7-96a7-dae80e5488d4 None admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:25.544352 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf318382-99c9-40e7-96a7-dae80e5488d4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:25.544714 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf318382-99c9-40e7-96a7-dae80e5488d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:25.546085 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf318382-99c9-40e7-96a7-dae80e5488d4 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:25.555289 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 413/822] 10.222.0.15 () {56 vars in 1099 bytes} [Sun Sep 11 10:34:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:25.560983 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e311b4e-5d14-4cee-ac91-fc67a47774bb None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:25.666449 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e311b4e-5d14-4cee-ac91-fc67a47774bb None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:25.702821 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60512b5f-b41f-4854-bfbd-2f63b100f5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:25.703486 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60512b5f-b41f-4854-bfbd-2f63b100f5e8 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:25.703771 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60512b5f-b41f-4854-bfbd-2f63b100f5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:25.704045 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60512b5f-b41f-4854-bfbd-2f63b100f5e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:25.716538 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60512b5f-b41f-4854-bfbd-2f63b100f5e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:25.718051 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60512b5f-b41f-4854-bfbd-2f63b100f5e8 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:25.733280 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-60512b5f-b41f-4854-bfbd-2f63b100f5e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:25.733280 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:25.733280 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:25.733280 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:25.733280 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:25.733280 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:25.733280 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:25.733280 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:25.733280 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:25.733280 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:25.733280 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:25.733280 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:25.733280 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:34:25.733280 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:34:25.733280 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:34:25.733280 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:34:25.734236 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:25.734236 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:25.734236 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:25.734236 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:25.734236 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:25.734236 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:25.734236 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:25.734236 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:25.734236 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:25.734236 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:25.734236 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:25.734236 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:25.734236 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:25.734236 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:25.734236 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:25.734236 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:25.734236 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:34:25.735325 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:34:25.735325 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:25.735325 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:25.735325 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 410/823] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:34:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:25.740303 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf318382-99c9-40e7-96a7-dae80e5488d4 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:25.847416 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf318382-99c9-40e7-96a7-dae80e5488d4 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:25.886388 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa004a1-7a52-470f-8a5f-94da2e49b894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:25.887008 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa004a1-7a52-470f-8a5f-94da2e49b894 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:25.887285 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa004a1-7a52-470f-8a5f-94da2e49b894 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:25.887586 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa004a1-7a52-470f-8a5f-94da2e49b894 None admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:25.888226 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa004a1-7a52-470f-8a5f-94da2e49b894 None admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:25.888526 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa004a1-7a52-470f-8a5f-94da2e49b894 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:25.889960 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa004a1-7a52-470f-8a5f-94da2e49b894 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:25.903197 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 414/824] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:34:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:25.908643 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60512b5f-b41f-4854-bfbd-2f63b100f5e8 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:25.961300 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:26.013629 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60512b5f-b41f-4854-bfbd-2f63b100f5e8 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:26.048283 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bee207-e0f1-4d26-80ec-7598d41afc5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:26.048920 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bee207-e0f1-4d26-80ec-7598d41afc5c None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:26.049204 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bee207-e0f1-4d26-80ec-7598d41afc5c None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:26.049477 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bee207-e0f1-4d26-80ec-7598d41afc5c None admin] PATH_INFO: `/v3/users/nova` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:26.057636 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bee207-e0f1-4d26-80ec-7598d41afc5c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:26.059076 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bee207-e0f1-4d26-80ec-7598d41afc5c None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:26.070400 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-d6bee207-e0f1-4d26-80ec-7598d41afc5c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 11 10:34:26.070400 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:26.070400 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:26.070400 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:26.070400 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:26.070400 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:26.070400 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:26.070400 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:26.070400 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:26.070400 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:26.070400 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:26.070400 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:26.070400 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 10:34:26.070400 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 10:34:26.070400 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 10:34:26.070400 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 10:34:26.072516 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:26.072516 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:26.072516 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 10:34:26.072516 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:34:26.072516 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 10:34:26.072516 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:34:26.072516 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:26.072516 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:26.072516 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:26.072516 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:26.072516 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:26.072516 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:26.072516 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:26.072516 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:26.072516 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:26.072516 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:26.072516 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:26.072516 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:26.074818 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:26.074818 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:26.074818 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 10:34:26.074818 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 10:34:26.074818 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 10:34:26.074818 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 10:34:26.074818 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 10:34:26.074818 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 10:34:26.074818 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 11 10:34:26.074818 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:26.076102 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 411/825] 10.222.0.15 () {56 vars in 1093 bytes} [Sun Sep 11 10:34:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:26.078943 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa004a1-7a52-470f-8a5f-94da2e49b894 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:26.121178 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:26.186858 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa004a1-7a52-470f-8a5f-94da2e49b894 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:26.229303 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d93fbe7-9670-45f7-a5d4-b353f1c2ddf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:26.229909 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93fbe7-9670-45f7-a5d4-b353f1c2ddf9 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:26.230186 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93fbe7-9670-45f7-a5d4-b353f1c2ddf9 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:26.230510 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93fbe7-9670-45f7-a5d4-b353f1c2ddf9 None admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:26.231232 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d93fbe7-9670-45f7-a5d4-b353f1c2ddf9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:26.231560 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d93fbe7-9670-45f7-a5d4-b353f1c2ddf9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:26.232969 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d93fbe7-9670-45f7-a5d4-b353f1c2ddf9 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:26.251761 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 415/826] 10.222.0.15 () {56 vars in 1133 bytes} [Sun Sep 11 10:34:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:26.257783 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bee207-e0f1-4d26-80ec-7598d41afc5c None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:26.367119 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bee207-e0f1-4d26-80ec-7598d41afc5c None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:26.377900 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:26.402896 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfc1e9f-36a8-4f5c-a1e1-ca5c19feb916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:26.403527 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfc1e9f-36a8-4f5c-a1e1-ca5c19feb916 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:26.403813 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfc1e9f-36a8-4f5c-a1e1-ca5c19feb916 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:26.404084 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfc1e9f-36a8-4f5c-a1e1-ca5c19feb916 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:26.473361 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfc1e9f-36a8-4f5c-a1e1-ca5c19feb916 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:26.475334 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfc1e9f-36a8-4f5c-a1e1-ca5c19feb916 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:26.576587 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 412/827] 10.222.0.15 () {58 vars in 1282 bytes} [Sun Sep 11 10:34:26 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 10:34:26.581976 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d93fbe7-9670-45f7-a5d4-b353f1c2ddf9 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:26.683881 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:26.689509 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d93fbe7-9670-45f7-a5d4-b353f1c2ddf9 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:26.727947 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e23279-355c-4b18-b8df-01c230b5af77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:26.728799 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e23279-355c-4b18-b8df-01c230b5af77 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:26.729145 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e23279-355c-4b18-b8df-01c230b5af77 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:26.729481 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e23279-355c-4b18-b8df-01c230b5af77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:26.742436 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e23279-355c-4b18-b8df-01c230b5af77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:26.744504 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e23279-355c-4b18-b8df-01c230b5af77 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:26.761364 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-e6e23279-355c-4b18-b8df-01c230b5af77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:26.761364 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:26.761364 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:26.761364 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:26.761364 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:26.761364 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:26.761364 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:26.761364 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:26.761364 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:26.761364 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:26.761364 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:26.761364 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:26.761364 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:34:26.761364 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:34:26.761364 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:34:26.761364 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:34:26.763475 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:26.763475 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:26.763475 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:26.763475 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:26.763475 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:26.763475 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:26.763475 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:26.763475 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:26.763475 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:26.763475 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:26.763475 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:26.763475 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:26.763475 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:26.763475 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:26.763475 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:26.763475 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:26.763475 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:34:26.765598 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:34:26.765598 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:26.765598 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:26.765598 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 416/828] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:34:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:26.769161 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfc1e9f-36a8-4f5c-a1e1-ca5c19feb916 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:26.781853 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:26.864945 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947743c7-1f54-44b6-87e8-98b5728a9e7f None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:26.866155 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947743c7-1f54-44b6-87e8-98b5728a9e7f None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:26.866155 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947743c7-1f54-44b6-87e8-98b5728a9e7f None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:26.867930 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 417/829] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:34:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 10:34:26.873228 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93df8f5-d062-4298-b23c-8724ee143b3d None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:26.873517 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93df8f5-d062-4298-b23c-8724ee143b3d None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:26.873798 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93df8f5-d062-4298-b23c-8724ee143b3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:26.889878 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfc1e9f-36a8-4f5c-a1e1-ca5c19feb916 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:26.893276 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-a93df8f5-d062-4298-b23c-8724ee143b3d None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:34:26.894368 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-a93df8f5-d062-4298-b23c-8724ee143b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:34:26.929821 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0855d8c0-8522-44b9-8eb3-606f32d1e4a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:26.930772 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0855d8c0-8522-44b9-8eb3-606f32d1e4a8 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:26.931243 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0855d8c0-8522-44b9-8eb3-606f32d1e4a8 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:26.931724 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0855d8c0-8522-44b9-8eb3-606f32d1e4a8 None admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:26.932572 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0855d8c0-8522-44b9-8eb3-606f32d1e4a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:26.933144 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0855d8c0-8522-44b9-8eb3-606f32d1e4a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:26.935078 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0855d8c0-8522-44b9-8eb3-606f32d1e4a8 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:26.952659 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 413/830] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:34:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:26.954181 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0855d8c0-8522-44b9-8eb3-606f32d1e4a8 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:26.979526 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 418/831] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:34:26 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 10:34:27.040374 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:27.065276 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0855d8c0-8522-44b9-8eb3-606f32d1e4a8 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:27.101655 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba6a2cc-98a2-4121-88df-db09c801e8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:27.102575 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba6a2cc-98a2-4121-88df-db09c801e8ee None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:27.102858 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba6a2cc-98a2-4121-88df-db09c801e8ee None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:27.103133 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba6a2cc-98a2-4121-88df-db09c801e8ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:27.111197 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba6a2cc-98a2-4121-88df-db09c801e8ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:27.112721 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba6a2cc-98a2-4121-88df-db09c801e8ee None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:27.125952 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-aba6a2cc-98a2-4121-88df-db09c801e8ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:34:27.125952 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:27.125952 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:27.125952 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:27.125952 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:27.125952 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:27.125952 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:27.125952 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:27.125952 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:27.125952 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:27.125952 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:27.125952 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:27.125952 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 10:34:27.125952 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 10:34:27.125952 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 10:34:27.127011 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 10:34:27.127011 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:27.127011 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:27.127011 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:27.127011 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:27.127011 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:27.127011 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:27.127011 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:27.127011 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:27.127011 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:27.127011 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:27.127011 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:27.127011 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:27.127011 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:27.127011 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:27.127011 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:27.127011 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:27.128033 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 10:34:27.128033 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 10:34:27.128033 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 10:34:27.128033 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 10:34:27.128033 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 10:34:27.128033 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 10:34:27.128033 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:34:27.128033 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:27.131830 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 414/832] 10.222.0.15 () {56 vars in 1111 bytes} [Sun Sep 11 10:34:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 10:34:27.134225 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93df8f5-d062-4298-b23c-8724ee143b3d None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:27.224186 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:27.224518 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9498e50e-6c74-4bfc-8eb5-48ce5747973f None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:27.224853 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9498e50e-6c74-4bfc-8eb5-48ce5747973f None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:27.225220 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9498e50e-6c74-4bfc-8eb5-48ce5747973f None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:27.227031 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 415/833] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:34:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:34:27.235450 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925942f0-3119-4727-85be-715d2b65e8d5 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:27.235450 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925942f0-3119-4727-85be-715d2b65e8d5 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:27.235450 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925942f0-3119-4727-85be-715d2b65e8d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:27.250875 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93df8f5-d062-4298-b23c-8724ee143b3d None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:27.252799 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-925942f0-3119-4727-85be-715d2b65e8d5 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:34:27.254313 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-925942f0-3119-4727-85be-715d2b65e8d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:34:27.297551 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073f723c-56c9-4938-ac74-427507698b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:27.298171 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f723c-56c9-4938-ac74-427507698b60 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:27.298453 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f723c-56c9-4938-ac74-427507698b60 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:27.298727 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f723c-56c9-4938-ac74-427507698b60 None admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:27.300266 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073f723c-56c9-4938-ac74-427507698b60 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:27.300580 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073f723c-56c9-4938-ac74-427507698b60 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:27.302007 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073f723c-56c9-4938-ac74-427507698b60 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:27.315479 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 419/834] 10.222.0.15 () {56 vars in 1148 bytes} [Sun Sep 11 10:34:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:27.316283 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073f723c-56c9-4938-ac74-427507698b60 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:27.341609 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 416/835] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:34:27 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 10:34:27.342926 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925942f0-3119-4727-85be-715d2b65e8d5 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:27.423094 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:27.424464 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073f723c-56c9-4938-ac74-427507698b60 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:27.468346 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0b36af-28e7-460d-a64e-c3d3c672d6f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:27.469274 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0b36af-28e7-460d-a64e-c3d3c672d6f8 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:27.469715 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0b36af-28e7-460d-a64e-c3d3c672d6f8 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:27.470137 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0b36af-28e7-460d-a64e-c3d3c672d6f8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:27.483332 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0b36af-28e7-460d-a64e-c3d3c672d6f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:34:27.487472 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925942f0-3119-4727-85be-715d2b65e8d5 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:27.491616 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:27.496529 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-0a0b36af-28e7-460d-a64e-c3d3c672d6f8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 10:34:27.496529 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:27.496529 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:27.496529 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:27.496529 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:27.496529 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:27.496529 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:27.496529 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:27.496529 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:27.496529 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:27.496529 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:27.496529 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:27.496529 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:34:27.496529 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:34:27.496529 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:34:27.496529 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:34:27.497484 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:34:27.497484 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:34:27.497484 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:34:27.497484 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:34:27.497484 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:27.497484 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:27.497484 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:27.497484 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:27.497484 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:27.497484 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:27.497484 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:27.497484 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:27.497484 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:27.497484 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:27.497484 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:27.497484 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:27.498335 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:27.498335 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:27.498335 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:27.498335 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:27.498335 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:34:27.498335 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:34:27.498335 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:34:27.498335 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:34:27.498335 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:34:27.498335 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:34:27.498335 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 10:34:27.498335 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:27.500977 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 420/836] 10.222.0.15 () {56 vars in 1096 bytes} [Sun Sep 11 10:34:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:27.507029 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0b36af-28e7-460d-a64e-c3d3c672d6f8 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:27.541119 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a3ea4f-c151-4f0a-8158-150a24e24b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:27.541871 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a3ea4f-c151-4f0a-8158-150a24e24b09 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:27.542160 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a3ea4f-c151-4f0a-8158-150a24e24b09 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:27.542435 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a3ea4f-c151-4f0a-8158-150a24e24b09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:27.543164 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a3ea4f-c151-4f0a-8158-150a24e24b09 None admin] RBAC: Adding query filter params (role.id=2cb7d73f0ea8414296367573f6902340, scope.project.id=5e1f845f99f04202b83621cb65e0adc2, user.id=2bd5356829b6454993bab9bad2f79fed) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:27.543536 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a3ea4f-c151-4f0a-8158-150a24e24b09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:27.545755 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a3ea4f-c151-4f0a-8158-150a24e24b09 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:27.555543 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 417/837] 10.222.0.15 () {56 vars in 1374 bytes} [Sun Sep 11 10:34:27 2022] GET /identity/v3/role_assignments?user.id=2bd5356829b6454993bab9bad2f79fed&scope.project.id=5e1f845f99f04202b83621cb65e0adc2&role.id=2cb7d73f0ea8414296367573f6902340 => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:27.561918 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3fa92d-fbee-49cb-b677-adc7a3146727 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:27.562255 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3fa92d-fbee-49cb-b677-adc7a3146727 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:27.562466 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3fa92d-fbee-49cb-b677-adc7a3146727 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:27.568264 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 418/838] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:34:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:34:27.570158 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9921be07-2b04-48de-a7a2-a76d112acf18 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:27.570444 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9921be07-2b04-48de-a7a2-a76d112acf18 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:27.570717 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9921be07-2b04-48de-a7a2-a76d112acf18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:27.594263 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-9921be07-2b04-48de-a7a2-a76d112acf18 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:34:27.595826 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-9921be07-2b04-48de-a7a2-a76d112acf18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:34:27.633401 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0b36af-28e7-460d-a64e-c3d3c672d6f8 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:27.672168 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8266a68c-d07c-4883-935a-c9a5f47f6150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:27.672793 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8266a68c-d07c-4883-935a-c9a5f47f6150 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:27.673075 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8266a68c-d07c-4883-935a-c9a5f47f6150 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:27.673347 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8266a68c-d07c-4883-935a-c9a5f47f6150 None admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:27.674017 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8266a68c-d07c-4883-935a-c9a5f47f6150 None admin] RBAC: Adding query filter params (name=admin) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:27.674319 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8266a68c-d07c-4883-935a-c9a5f47f6150 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:27.674983 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 419/839] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:34:27 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 10:34:27.675744 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8266a68c-d07c-4883-935a-c9a5f47f6150 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:27.676559 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9921be07-2b04-48de-a7a2-a76d112acf18 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:27.693061 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 421/840] 10.222.0.15 () {56 vars in 1099 bytes} [Sun Sep 11 10:34:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:27.694169 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8266a68c-d07c-4883-935a-c9a5f47f6150 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:27.761978 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:27.765153 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:27.815217 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8266a68c-d07c-4883-935a-c9a5f47f6150 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:27.826989 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9921be07-2b04-48de-a7a2-a76d112acf18 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:27.862101 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb66f07-659d-49b1-9657-90a670a8b56b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:27.863222 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb66f07-659d-49b1-9657-90a670a8b56b None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:27.863781 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb66f07-659d-49b1-9657-90a670a8b56b None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:27.864299 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb66f07-659d-49b1-9657-90a670a8b56b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:27.872010 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82585452-0e1f-4cf1-a7b2-f826596dc6b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:27.873181 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82585452-0e1f-4cf1-a7b2-f826596dc6b6 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:27.873462 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82585452-0e1f-4cf1-a7b2-f826596dc6b6 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:27.873735 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82585452-0e1f-4cf1-a7b2-f826596dc6b6 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:27.874321 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82585452-0e1f-4cf1-a7b2-f826596dc6b6 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:27.876202 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82585452-0e1f-4cf1-a7b2-f826596dc6b6 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:27.877523 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb66f07-659d-49b1-9657-90a670a8b56b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:27.879525 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb66f07-659d-49b1-9657-90a670a8b56b None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:27.886856 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-82585452-0e1f-4cf1-a7b2-f826596dc6b6 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 11 10:34:27.886856 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:27.886856 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 10:34:27.886856 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 10:34:27.886856 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 10:34:27.886856 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 10:34:27.886856 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 10:34:27.886856 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 10:34:27.886856 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 11 10:34:27.886856 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:27.886856 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 10:34:27.886856 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:27.886856 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:27.886856 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:27.886856 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:27.886856 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:27.886856 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:27.888137 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:27.888137 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:27.888137 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:27.888137 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:27.888137 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:27.888137 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:27.888137 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 10:34:27.888137 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 10:34:27.888137 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 10:34:27.888137 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 10:34:27.888137 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:27.888137 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:27.888137 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:27.888137 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:27.888137 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:27.888137 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:27.889491 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:27.889491 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:27.889491 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:27.889491 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:27.889491 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:27.889491 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:27.889491 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:27.889491 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:27.889491 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:27.889491 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:27.889491 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 10:34:27.889491 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 10:34:27.889491 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 11 10:34:27.889491 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:27.889491 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 420/841] 10.222.0.15 () {56 vars in 1114 bytes} [Sun Sep 11 10:34:27 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:27.897943 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82585452-0e1f-4cf1-a7b2-f826596dc6b6 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:27.902910 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-1bb66f07-659d-49b1-9657-90a670a8b56b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:27.902910 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:27.902910 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:27.902910 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:27.902910 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:27.902910 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:27.902910 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:27.902910 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:27.902910 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:27.902910 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:27.902910 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:27.902910 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:27.902910 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:34:27.902910 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:34:27.902910 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:34:27.902910 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:34:27.903991 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:27.903991 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:27.903991 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:27.903991 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:27.903991 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:27.903991 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:27.903991 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:27.903991 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:27.903991 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:27.903991 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:27.903991 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:27.903991 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:27.903991 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:27.903991 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:27.903991 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:27.903991 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:27.903991 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:34:27.905236 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:34:27.905236 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:27.905236 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:27.905236 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 422/842] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:34:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:27.912851 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb66f07-659d-49b1-9657-90a670a8b56b None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:28.012735 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82585452-0e1f-4cf1-a7b2-f826596dc6b6 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:28.029890 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:28.034822 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb66f07-659d-49b1-9657-90a670a8b56b None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:28.048500 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed24d28f-2649-4f89-8bb4-cf628e4dbb71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:28.049193 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:28.049305 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed24d28f-2649-4f89-8bb4-cf628e4dbb71 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:28.049543 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed24d28f-2649-4f89-8bb4-cf628e4dbb71 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:28.049817 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed24d28f-2649-4f89-8bb4-cf628e4dbb71 None admin] PATH_INFO: `/v3/services` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:28.050443 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed24d28f-2649-4f89-8bb4-cf628e4dbb71 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:28.050785 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed24d28f-2649-4f89-8bb4-cf628e4dbb71 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:28.052679 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed24d28f-2649-4f89-8bb4-cf628e4dbb71 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:28.058591 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 421/843] 10.222.0.15 () {56 vars in 1114 bytes} [Sun Sep 11 10:34:27 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:34:28.059380 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed24d28f-2649-4f89-8bb4-cf628e4dbb71 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:28.075005 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811701e9-ae7c-47c2-b9c0-6291f8be25bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:28.075976 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811701e9-ae7c-47c2-b9c0-6291f8be25bb None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:28.076476 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811701e9-ae7c-47c2-b9c0-6291f8be25bb None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:28.076941 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811701e9-ae7c-47c2-b9c0-6291f8be25bb None admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:28.077756 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811701e9-ae7c-47c2-b9c0-6291f8be25bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:28.078411 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811701e9-ae7c-47c2-b9c0-6291f8be25bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:28.080710 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811701e9-ae7c-47c2-b9c0-6291f8be25bb None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:28.100811 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 423/844] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:34:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:28.101930 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811701e9-ae7c-47c2-b9c0-6291f8be25bb None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:28.175963 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed24d28f-2649-4f89-8bb4-cf628e4dbb71 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:28.211610 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf12545-9f35-4928-9fcc-a9c4b075ac7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:28.212243 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf12545-9f35-4928-9fcc-a9c4b075ac7f None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:28.212524 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf12545-9f35-4928-9fcc-a9c4b075ac7f None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:28.212795 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf12545-9f35-4928-9fcc-a9c4b075ac7f None admin] PATH_INFO: `/v3/roles/service` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:28.213981 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811701e9-ae7c-47c2-b9c0-6291f8be25bb None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:28.223241 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf12545-9f35-4928-9fcc-a9c4b075ac7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:34:28.229025 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-cdf12545-9f35-4928-9fcc-a9c4b075ac7f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 10:34:28.229025 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:28.229025 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:28.229025 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:28.229025 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:28.229025 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:28.229025 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:28.229025 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:28.229025 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:28.229025 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:28.229025 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:28.229025 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:28.229025 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:34:28.229025 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:34:28.229025 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:34:28.229025 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:34:28.229982 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:34:28.229982 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:34:28.229982 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:34:28.229982 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:34:28.229982 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:28.229982 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:28.229982 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:28.229982 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:28.229982 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:28.229982 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:28.229982 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:28.229982 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:28.229982 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:28.229982 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:28.229982 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:28.229982 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:28.230827 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:28.230827 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:28.230827 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:28.230827 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:28.230827 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:34:28.230827 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:34:28.230827 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:34:28.230827 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:34:28.230827 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:34:28.230827 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:34:28.230827 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 10:34:28.230827 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:28.230827 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 422/845] 10.222.0.15 () {56 vars in 1102 bytes} [Sun Sep 11 10:34:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:28.231584 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf12545-9f35-4928-9fcc-a9c4b075ac7f None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:28.257468 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc6ca06-150f-489d-8317-c79c1b35340b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:28.258082 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc6ca06-150f-489d-8317-c79c1b35340b None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:28.258412 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc6ca06-150f-489d-8317-c79c1b35340b None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:28.258688 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc6ca06-150f-489d-8317-c79c1b35340b None admin] PATH_INFO: `/v3/services` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:28.259317 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc6ca06-150f-489d-8317-c79c1b35340b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:28.259649 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc6ca06-150f-489d-8317-c79c1b35340b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:28.261087 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc6ca06-150f-489d-8317-c79c1b35340b None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:28.275452 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 424/846] 10.222.0.15 () {56 vars in 1114 bytes} [Sun Sep 11 10:34:28 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:28.275452 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc6ca06-150f-489d-8317-c79c1b35340b None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:28.301300 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:28.312687 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:28.386338 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf12545-9f35-4928-9fcc-a9c4b075ac7f None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:28.415155 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc6ca06-150f-489d-8317-c79c1b35340b None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:28.424649 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdf62ea-56d9-4fe5-be79-7174ca202072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:28.425478 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf62ea-56d9-4fe5-be79-7174ca202072 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:28.426053 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf62ea-56d9-4fe5-be79-7174ca202072 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:28.426403 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf62ea-56d9-4fe5-be79-7174ca202072 None admin] PATH_INFO: `/v3/users/swift` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:28.440748 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdf62ea-56d9-4fe5-be79-7174ca202072 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:28.443230 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdf62ea-56d9-4fe5-be79-7174ca202072 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:28.460258 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-4bdf62ea-56d9-4fe5-be79-7174ca202072 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 11 10:34:28.460258 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:28.460258 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:28.460258 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:28.460258 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:28.460258 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:28.460258 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:28.460258 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:28.460258 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:28.460258 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:28.460258 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:28.460258 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:28.460258 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 10:34:28.460258 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 10:34:28.460258 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 10:34:28.460258 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 10:34:28.461297 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:28.461297 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:28.461297 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 10:34:28.461297 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:34:28.461297 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 10:34:28.461297 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:34:28.461297 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:28.461297 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:28.461297 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:28.461297 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:28.461297 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:28.461297 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:28.461297 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:28.461297 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:28.461297 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:28.461297 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:28.461297 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:28.461297 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:28.462245 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:28.462245 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:28.462245 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 10:34:28.462245 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 10:34:28.462245 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 10:34:28.462245 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 10:34:28.462245 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 10:34:28.462245 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 10:34:28.462245 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 11 10:34:28.462245 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:28.462805 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdba6cd7-9d7a-4181-a2a0-0da51c5d3cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:28.462805 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba6cd7-9d7a-4181-a2a0-0da51c5d3cfc None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:28.462959 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 423/847] 10.222.0.15 () {56 vars in 1096 bytes} [Sun Sep 11 10:34:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:28.463036 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba6cd7-9d7a-4181-a2a0-0da51c5d3cfc None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:28.463036 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba6cd7-9d7a-4181-a2a0-0da51c5d3cfc None admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:28.463036 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdba6cd7-9d7a-4181-a2a0-0da51c5d3cfc None admin] RBAC: Adding query filter params (name=service) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:28.463036 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdba6cd7-9d7a-4181-a2a0-0da51c5d3cfc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:28.464312 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdba6cd7-9d7a-4181-a2a0-0da51c5d3cfc None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:28.469043 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdf62ea-56d9-4fe5-be79-7174ca202072 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:28.478662 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 425/848] 10.222.0.15 () {56 vars in 1103 bytes} [Sun Sep 11 10:34:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:28.484183 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdba6cd7-9d7a-4181-a2a0-0da51c5d3cfc None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:28.594417 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdf62ea-56d9-4fe5-be79-7174ca202072 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:28.596260 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:28.600750 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:28.605671 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdba6cd7-9d7a-4181-a2a0-0da51c5d3cfc None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:28.641608 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66beb0cd-10f0-489b-9dbb-c497bec723ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:28.642399 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66beb0cd-10f0-489b-9dbb-c497bec723ed None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:28.642716 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0429f93d-2b3e-4c2a-8caf-2cb93717021e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:28.642856 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66beb0cd-10f0-489b-9dbb-c497bec723ed None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:28.643222 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66beb0cd-10f0-489b-9dbb-c497bec723ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:28.643328 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0429f93d-2b3e-4c2a-8caf-2cb93717021e None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:28.643634 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0429f93d-2b3e-4c2a-8caf-2cb93717021e None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:28.644235 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0429f93d-2b3e-4c2a-8caf-2cb93717021e None admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:28.644972 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0429f93d-2b3e-4c2a-8caf-2cb93717021e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:28.645280 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0429f93d-2b3e-4c2a-8caf-2cb93717021e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:28.647892 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0429f93d-2b3e-4c2a-8caf-2cb93717021e None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:28.655043 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66beb0cd-10f0-489b-9dbb-c497bec723ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:28.656618 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66beb0cd-10f0-489b-9dbb-c497bec723ed None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:28.671582 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 424/849] 10.222.0.15 () {56 vars in 1135 bytes} [Sun Sep 11 10:34:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:28.677035 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-66beb0cd-10f0-489b-9dbb-c497bec723ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:28.677035 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:28.677035 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:28.677035 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:28.677035 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:28.677035 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:28.677035 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:28.677035 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:28.677035 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:28.677035 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:28.677035 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:28.677035 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:28.677035 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:34:28.677035 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:34:28.677035 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:34:28.677035 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:34:28.678240 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:28.678240 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:28.678240 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:28.678240 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:28.678240 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:28.678240 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:28.678240 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:28.678240 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:28.678240 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:28.678240 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:28.678240 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:28.678240 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:28.678240 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:28.678240 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:28.678240 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:28.678240 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:28.678240 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:34:28.679168 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:34:28.679168 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:28.679168 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:28.679340 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0429f93d-2b3e-4c2a-8caf-2cb93717021e None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:28.679439 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 426/850] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:34:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 10:34:28.684324 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66beb0cd-10f0-489b-9dbb-c497bec723ed None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:28.796634 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0429f93d-2b3e-4c2a-8caf-2cb93717021e None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:28.807484 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66beb0cd-10f0-489b-9dbb-c497bec723ed None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:28.837453 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a5ec49-60c8-4af3-804d-92d0ca851f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:28.838156 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a5ec49-60c8-4af3-804d-92d0ca851f98 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:28.838433 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a5ec49-60c8-4af3-804d-92d0ca851f98 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:28.838706 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a5ec49-60c8-4af3-804d-92d0ca851f98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:28.840971 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee86ceb-c2c1-41ab-acf6-46753f3a8616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:28.841581 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee86ceb-c2c1-41ab-acf6-46753f3a8616 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:28.841859 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee86ceb-c2c1-41ab-acf6-46753f3a8616 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:28.842131 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee86ceb-c2c1-41ab-acf6-46753f3a8616 None admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:28.842765 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee86ceb-c2c1-41ab-acf6-46753f3a8616 None admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:28.843063 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee86ceb-c2c1-41ab-acf6-46753f3a8616 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:28.844576 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee86ceb-c2c1-41ab-acf6-46753f3a8616 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:28.857309 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 427/851] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:34:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:28.862736 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee86ceb-c2c1-41ab-acf6-46753f3a8616 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:28.876191 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:28.877764 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:28.935249 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a5ec49-60c8-4af3-804d-92d0ca851f98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:28.937355 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a5ec49-60c8-4af3-804d-92d0ca851f98 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:28.985519 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee86ceb-c2c1-41ab-acf6-46753f3a8616 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:29.030349 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a375e9b-12f1-4955-9cc3-e064876fa0ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:29.031237 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a375e9b-12f1-4955-9cc3-e064876fa0ca None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:29.031889 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a375e9b-12f1-4955-9cc3-e064876fa0ca None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:29.032226 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a375e9b-12f1-4955-9cc3-e064876fa0ca None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:29.040717 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a375e9b-12f1-4955-9cc3-e064876fa0ca None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:29.042982 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a375e9b-12f1-4955-9cc3-e064876fa0ca None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:29.051588 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 425/852] 10.222.0.15 () {58 vars in 1282 bytes} [Sun Sep 11 10:34:28 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 10:34:29.055207 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a5ec49-60c8-4af3-804d-92d0ca851f98 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:29.063219 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-8a375e9b-12f1-4955-9cc3-e064876fa0ca None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 10:34:29.063219 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:29.063219 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:29.063219 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:29.063219 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:29.063219 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:29.063219 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:29.063219 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:29.063219 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:29.063219 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:29.063219 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:29.063219 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:29.063219 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 10:34:29.063219 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 10:34:29.063219 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 10:34:29.063219 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 10:34:29.064216 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:29.064216 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:29.064216 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 10:34:29.064216 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:34:29.064216 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 10:34:29.064216 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:34:29.064216 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:29.064216 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:29.064216 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:29.064216 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:29.064216 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:29.064216 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:29.064216 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:29.064216 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:29.064216 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:29.064216 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:29.064216 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:29.064216 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:29.065197 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:29.065197 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:29.065197 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 10:34:29.065197 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 10:34:29.065197 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 10:34:29.065197 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 10:34:29.065197 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 10:34:29.065197 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 10:34:29.065197 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 10:34:29.065197 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:29.069942 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 428/853] 10.222.0.15 () {56 vars in 1117 bytes} [Sun Sep 11 10:34:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:29.071382 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a375e9b-12f1-4955-9cc3-e064876fa0ca None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:29.146157 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:29.155247 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:29.184792 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a5ec49-60c8-4af3-804d-92d0ca851f98 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:29.189448 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a375e9b-12f1-4955-9cc3-e064876fa0ca None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:29.221047 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e865e968-a93d-464d-939a-4432e5f182a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:29.222106 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e865e968-a93d-464d-939a-4432e5f182a7 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:29.222533 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e865e968-a93d-464d-939a-4432e5f182a7 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:29.222974 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e865e968-a93d-464d-939a-4432e5f182a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:29.227981 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41061ef6-5cf5-41a3-be31-7eba97d800bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:29.228641 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41061ef6-5cf5-41a3-be31-7eba97d800bf None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:29.228924 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41061ef6-5cf5-41a3-be31-7eba97d800bf None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:29.229194 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41061ef6-5cf5-41a3-be31-7eba97d800bf None admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:29.229950 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41061ef6-5cf5-41a3-be31-7eba97d800bf None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:29.230254 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41061ef6-5cf5-41a3-be31-7eba97d800bf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:29.231771 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41061ef6-5cf5-41a3-be31-7eba97d800bf None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:29.236798 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e865e968-a93d-464d-939a-4432e5f182a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:29.238465 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e865e968-a93d-464d-939a-4432e5f182a7 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:29.255576 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-e865e968-a93d-464d-939a-4432e5f182a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:29.255576 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:29.255576 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:29.255576 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:29.255576 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:29.255576 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:29.255576 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:29.255576 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:29.255576 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:29.255576 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:29.255576 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:29.255576 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:29.255576 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:34:29.255576 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:34:29.255576 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:34:29.255576 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:34:29.257478 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:29.257478 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:29.257478 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:29.257478 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:29.257478 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:29.257478 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:29.257478 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:29.257478 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:29.257478 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:29.257478 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:29.257478 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:29.257478 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:29.257478 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:29.257478 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:29.257478 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:29.257478 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:29.257478 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:34:29.258403 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:34:29.258403 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:29.258403 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:29.258841 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 429/854] 10.222.0.15 () {56 vars in 1149 bytes} [Sun Sep 11 10:34:29 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:34:29.259826 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41061ef6-5cf5-41a3-be31-7eba97d800bf None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:29.260501 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 426/855] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:34:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:29.267485 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e865e968-a93d-464d-939a-4432e5f182a7 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:29.374075 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41061ef6-5cf5-41a3-be31-7eba97d800bf None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:29.396568 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e865e968-a93d-464d-939a-4432e5f182a7 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:29.401868 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:29.409925 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd4511d-a4da-4a3a-b51a-a7ec2b4a08d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:29.410454 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd4511d-a4da-4a3a-b51a-a7ec2b4a08d0 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:29.410736 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd4511d-a4da-4a3a-b51a-a7ec2b4a08d0 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:29.411004 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd4511d-a4da-4a3a-b51a-a7ec2b4a08d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:29.414076 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:29.437203 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acee8978-bccb-42a9-857a-a696570e86a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:29.438163 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee8978-bccb-42a9-857a-a696570e86a8 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:29.438665 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee8978-bccb-42a9-857a-a696570e86a8 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:29.439133 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee8978-bccb-42a9-857a-a696570e86a8 None admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:29.439996 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acee8978-bccb-42a9-857a-a696570e86a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:29.440535 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acee8978-bccb-42a9-857a-a696570e86a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:29.442092 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acee8978-bccb-42a9-857a-a696570e86a8 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:29.455784 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 427/856] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:34:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:29.461796 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acee8978-bccb-42a9-857a-a696570e86a8 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:29.492431 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd4511d-a4da-4a3a-b51a-a7ec2b4a08d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:29.494394 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd4511d-a4da-4a3a-b51a-a7ec2b4a08d0 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:29.573576 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acee8978-bccb-42a9-857a-a696570e86a8 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:29.607536 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 430/857] 10.222.0.15 () {58 vars in 1282 bytes} [Sun Sep 11 10:34:29 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 10:34:29.612251 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd4511d-a4da-4a3a-b51a-a7ec2b4a08d0 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:29.616994 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c21871-a05d-40b9-b9e6-313c35337b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:29.617743 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c21871-a05d-40b9-b9e6-313c35337b5c None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:29.618098 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c21871-a05d-40b9-b9e6-313c35337b5c None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:29.618440 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c21871-a05d-40b9-b9e6-313c35337b5c None admin] PATH_INFO: `/v3/projects/service` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:29.626219 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c21871-a05d-40b9-b9e6-313c35337b5c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:29.627790 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c21871-a05d-40b9-b9e6-313c35337b5c None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:29.640772 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-95c21871-a05d-40b9-b9e6-313c35337b5c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:34:29.640772 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:29.640772 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:29.640772 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:29.640772 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:29.640772 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:29.640772 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:29.640772 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:29.640772 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:29.640772 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:29.640772 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:29.640772 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:29.640772 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 10:34:29.640772 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 10:34:29.640772 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 10:34:29.641712 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 10:34:29.641712 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:29.641712 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:29.641712 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:29.641712 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:29.641712 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:29.641712 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:29.641712 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:29.641712 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:29.641712 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:29.641712 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:29.641712 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:29.641712 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:29.641712 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:29.641712 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:29.641712 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:29.641712 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:29.642608 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 10:34:29.642608 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 10:34:29.642608 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 10:34:29.642608 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 10:34:29.642608 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 10:34:29.642608 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 10:34:29.642608 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:34:29.642608 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:29.646956 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 428/858] 10.222.0.15 () {56 vars in 1111 bytes} [Sun Sep 11 10:34:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 10:34:29.647789 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:29.648881 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c21871-a05d-40b9-b9e6-313c35337b5c None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:29.682983 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:29.728382 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd4511d-a4da-4a3a-b51a-a7ec2b4a08d0 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:29.761317 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c21871-a05d-40b9-b9e6-313c35337b5c None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:29.774471 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248fc070-4b2e-4883-8ecb-f07ec189380c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:29.775789 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248fc070-4b2e-4883-8ecb-f07ec189380c None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:29.776252 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248fc070-4b2e-4883-8ecb-f07ec189380c None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:29.776748 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248fc070-4b2e-4883-8ecb-f07ec189380c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:29.790131 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248fc070-4b2e-4883-8ecb-f07ec189380c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:29.792107 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248fc070-4b2e-4883-8ecb-f07ec189380c None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:29.801099 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee23a75a-3921-4cb5-9959-235ecf59720c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:29.801832 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee23a75a-3921-4cb5-9959-235ecf59720c None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:29.802206 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee23a75a-3921-4cb5-9959-235ecf59720c None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:29.802551 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee23a75a-3921-4cb5-9959-235ecf59720c None admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:29.803925 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee23a75a-3921-4cb5-9959-235ecf59720c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:29.804335 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee23a75a-3921-4cb5-9959-235ecf59720c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:29.806003 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee23a75a-3921-4cb5-9959-235ecf59720c None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:29.811779 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-248fc070-4b2e-4883-8ecb-f07ec189380c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:29.811779 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:29.811779 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:29.811779 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:29.811779 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:29.811779 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:29.811779 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:29.811779 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:29.811779 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:29.811779 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:29.811779 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:29.811779 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:29.811779 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:34:29.811779 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:34:29.811779 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:34:29.811779 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:34:29.812781 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:29.812781 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:29.812781 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:29.812781 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:29.812781 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:29.812781 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:29.812781 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:29.812781 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:29.812781 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:29.812781 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:29.812781 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:29.812781 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:29.812781 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:29.812781 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:29.812781 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:29.812781 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:29.812781 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:34:29.813662 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:34:29.813662 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:29.813662 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:29.815817 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 431/859] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:34:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 10:34:29.822126 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248fc070-4b2e-4883-8ecb-f07ec189380c None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:29.824610 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 429/860] 10.222.0.15 () {56 vars in 1148 bytes} [Sun Sep 11 10:34:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:29.830331 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee23a75a-3921-4cb5-9959-235ecf59720c None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:29.922512 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:29.926617 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:29.939443 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee23a75a-3921-4cb5-9959-235ecf59720c None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:29.947677 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248fc070-4b2e-4883-8ecb-f07ec189380c None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:29.978057 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db9f38f-bc5b-4ab5-accb-18c79fa6516b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:29.978670 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db9f38f-bc5b-4ab5-accb-18c79fa6516b None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:29.978952 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db9f38f-bc5b-4ab5-accb-18c79fa6516b None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:29.979225 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db9f38f-bc5b-4ab5-accb-18c79fa6516b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:29.979974 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db9f38f-bc5b-4ab5-accb-18c79fa6516b None admin] RBAC: Adding query filter params (role.id=2cb7d73f0ea8414296367573f6902340, scope.project.id=5e1f845f99f04202b83621cb65e0adc2, user.id=cc7ca566e66a45b29e3d02f47bf84981) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:29.980303 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db9f38f-bc5b-4ab5-accb-18c79fa6516b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:29.981728 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db9f38f-bc5b-4ab5-accb-18c79fa6516b None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:29.987442 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 430/861] 10.222.0.15 () {56 vars in 1374 bytes} [Sun Sep 11 10:34:29 2022] GET /identity/v3/role_assignments?user.id=cc7ca566e66a45b29e3d02f47bf84981&scope.project.id=5e1f845f99f04202b83621cb65e0adc2&role.id=2cb7d73f0ea8414296367573f6902340 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:34:29.987962 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8562c6-e582-4ac4-ae9c-97ff856253d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:29.988632 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a0c9eb-20a8-4366-bac2-ebc0a5be350d None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:29.988921 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a0c9eb-20a8-4366-bac2-ebc0a5be350d None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:29.989193 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a0c9eb-20a8-4366-bac2-ebc0a5be350d None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:29.989647 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8562c6-e582-4ac4-ae9c-97ff856253d6 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:29.990085 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8562c6-e582-4ac4-ae9c-97ff856253d6 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:29.991108 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8562c6-e582-4ac4-ae9c-97ff856253d6 None admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:29.991338 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 431/862] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:34:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:34:29.992202 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8562c6-e582-4ac4-ae9c-97ff856253d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:29.992507 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8562c6-e582-4ac4-ae9c-97ff856253d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:29.993983 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8562c6-e582-4ac4-ae9c-97ff856253d6 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:29.997184 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78d5cf7-45cc-4041-a9da-9fb5268be7c1 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:29.997538 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78d5cf7-45cc-4041-a9da-9fb5268be7c1 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:29.997879 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78d5cf7-45cc-4041-a9da-9fb5268be7c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:30.013623 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 432/863] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:34:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:30.018484 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-e78d5cf7-45cc-4041-a9da-9fb5268be7c1 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:34:30.018990 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8562c6-e582-4ac4-ae9c-97ff856253d6 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:30.019859 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-e78d5cf7-45cc-4041-a9da-9fb5268be7c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:34:30.103902 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 432/864] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:34:29 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 10:34:30.142817 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8562c6-e582-4ac4-ae9c-97ff856253d6 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:30.182901 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4047453-e35a-4561-a520-94cf88946655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:30.183870 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4047453-e35a-4561-a520-94cf88946655 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:30.184325 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4047453-e35a-4561-a520-94cf88946655 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:30.184771 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4047453-e35a-4561-a520-94cf88946655 None admin] PATH_INFO: `/v3/projects/service` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:30.192183 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4047453-e35a-4561-a520-94cf88946655 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:30.194546 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4047453-e35a-4561-a520-94cf88946655 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:30.197813 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:30.208079 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-c4047453-e35a-4561-a520-94cf88946655 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:34:30.208079 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:30.208079 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:30.208079 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:30.208079 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:30.208079 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:30.208079 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:30.208079 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:30.208079 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:30.208079 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:30.208079 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:30.208079 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:30.208079 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 10:34:30.208079 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 10:34:30.208079 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 10:34:30.209778 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 10:34:30.209778 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:30.209778 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:30.209778 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:30.209778 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:30.209778 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:30.209778 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:30.209778 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:30.209778 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:30.209778 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:30.209778 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:30.209778 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:30.209778 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:30.209778 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:30.209778 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:30.209778 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:30.209778 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:30.210899 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 10:34:30.210899 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 10:34:30.210899 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 10:34:30.210899 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 10:34:30.210899 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 10:34:30.210899 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 10:34:30.210899 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:34:30.210899 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:30.212484 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 433/865] 10.222.0.15 () {56 vars in 1111 bytes} [Sun Sep 11 10:34:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:30.218136 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78d5cf7-45cc-4041-a9da-9fb5268be7c1 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:30.316217 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:30.332765 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78d5cf7-45cc-4041-a9da-9fb5268be7c1 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:30.369435 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2ad416-a606-4efd-a819-266d7cd7e84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:30.370193 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ad416-a606-4efd-a819-266d7cd7e84e None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:30.370569 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ad416-a606-4efd-a819-266d7cd7e84e None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:30.370937 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ad416-a606-4efd-a819-266d7cd7e84e None admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:30.371757 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2ad416-a606-4efd-a819-266d7cd7e84e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:30.372156 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2ad416-a606-4efd-a819-266d7cd7e84e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:30.373735 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2ad416-a606-4efd-a819-266d7cd7e84e None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:30.386292 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 433/866] 10.222.0.15 () {56 vars in 1148 bytes} [Sun Sep 11 10:34:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:30.389724 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4047453-e35a-4561-a520-94cf88946655 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:30.392188 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2ad416-a606-4efd-a819-266d7cd7e84e None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:30.499510 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4047453-e35a-4561-a520-94cf88946655 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:30.512812 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2ad416-a606-4efd-a819-266d7cd7e84e None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:30.536594 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a080c4d3-d96e-412b-9796-e778afd309fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:30.538181 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a080c4d3-d96e-412b-9796-e778afd309fe None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:30.538805 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a080c4d3-d96e-412b-9796-e778afd309fe None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:30.539227 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a080c4d3-d96e-412b-9796-e778afd309fe None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:30.539995 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a080c4d3-d96e-412b-9796-e778afd309fe None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:30.541520 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a080c4d3-d96e-412b-9796-e778afd309fe None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:30.555533 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-a080c4d3-d96e-412b-9796-e778afd309fe None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 11 10:34:30.555533 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:30.555533 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 10:34:30.555533 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 10:34:30.555533 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 10:34:30.555533 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 10:34:30.555533 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 10:34:30.555533 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 10:34:30.555533 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 11 10:34:30.555533 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:30.555533 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 10:34:30.555533 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:30.555533 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:30.555533 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:30.555533 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:30.555533 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:30.556709 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:30.556709 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:30.556709 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:30.556709 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:30.556709 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:30.556709 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:30.556709 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:30.556709 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 10:34:30.556709 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 10:34:30.556709 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 10:34:30.556709 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 10:34:30.556709 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:30.556709 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:30.556709 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:30.556709 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:30.556709 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:30.556709 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:30.557823 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:30.557823 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:30.557823 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:30.557823 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:30.557823 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:30.557823 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:30.557823 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:30.557823 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:30.557823 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:30.557823 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:30.557823 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 10:34:30.557823 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 10:34:30.557823 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 11 10:34:30.557823 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:30.557823 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 434/867] 10.222.0.15 () {56 vars in 1132 bytes} [Sun Sep 11 10:34:30 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 10:34:30.560253 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b90b8-2232-4c7f-b8a0-7bf09c0b4475 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:30.560538 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b90b8-2232-4c7f-b8a0-7bf09c0b4475 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:30.560806 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b90b8-2232-4c7f-b8a0-7bf09c0b4475 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:30.562134 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 435/868] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:34:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:34:30.565987 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4b9b74-916a-4a87-a0fa-552c414fb0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:30.566146 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790b90b8-2232-4c7f-b8a0-7bf09c0b4475 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:30.566240 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4b9b74-916a-4a87-a0fa-552c414fb0f9 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:30.566240 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4b9b74-916a-4a87-a0fa-552c414fb0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:30.566240 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4b9b74-916a-4a87-a0fa-552c414fb0f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:30.566240 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4b9b74-916a-4a87-a0fa-552c414fb0f9 None admin] RBAC: Adding query filter params (role.id=3cb7ebbc707d453586b3d0f4ab3cf103, scope.project.id=5e1f845f99f04202b83621cb65e0adc2, user.id=bb44e7aef0c7436d93a7233307b5f095) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:30.566240 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4b9b74-916a-4a87-a0fa-552c414fb0f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:30.566775 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4b9b74-916a-4a87-a0fa-552c414fb0f9 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:30.574903 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 434/869] 10.222.0.15 () {56 vars in 1374 bytes} [Sun Sep 11 10:34:30 2022] GET /identity/v3/role_assignments?user.id=bb44e7aef0c7436d93a7233307b5f095&scope.project.id=5e1f845f99f04202b83621cb65e0adc2&role.id=3cb7ebbc707d453586b3d0f4ab3cf103 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:30.579558 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62379b0d-5620-4368-aded-43791113eeaa None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:30.579558 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62379b0d-5620-4368-aded-43791113eeaa None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:30.579558 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62379b0d-5620-4368-aded-43791113eeaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:30.601301 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-62379b0d-5620-4368-aded-43791113eeaa None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:34:30.603548 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-62379b0d-5620-4368-aded-43791113eeaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:34:30.607752 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:30.648819 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:30.688803 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 435/870] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:34:30 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 10:34:30.698319 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790b90b8-2232-4c7f-b8a0-7bf09c0b4475 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:30.739194 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74027381-c8f5-440b-b9ad-13ef1757cdae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:30.739844 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74027381-c8f5-440b-b9ad-13ef1757cdae None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:30.740121 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74027381-c8f5-440b-b9ad-13ef1757cdae None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:30.740419 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74027381-c8f5-440b-b9ad-13ef1757cdae None admin] PATH_INFO: `/v3/services` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:30.741858 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74027381-c8f5-440b-b9ad-13ef1757cdae None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:30.742328 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74027381-c8f5-440b-b9ad-13ef1757cdae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:30.743730 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74027381-c8f5-440b-b9ad-13ef1757cdae None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:30.749278 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 436/871] 10.222.0.15 () {56 vars in 1126 bytes} [Sun Sep 11 10:34:30 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:30.754889 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62379b0d-5620-4368-aded-43791113eeaa None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:30.864713 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62379b0d-5620-4368-aded-43791113eeaa None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:30.901196 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec21c639-aef4-4da4-84d0-9fe2a12fdd94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:30.901812 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec21c639-aef4-4da4-84d0-9fe2a12fdd94 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:30.902087 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec21c639-aef4-4da4-84d0-9fe2a12fdd94 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:30.902353 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec21c639-aef4-4da4-84d0-9fe2a12fdd94 None admin] PATH_INFO: `/v3/services` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:30.902977 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec21c639-aef4-4da4-84d0-9fe2a12fdd94 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:30.903276 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec21c639-aef4-4da4-84d0-9fe2a12fdd94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:30.904712 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec21c639-aef4-4da4-84d0-9fe2a12fdd94 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:30.909782 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 436/872] 10.222.0.15 () {56 vars in 1126 bytes} [Sun Sep 11 10:34:30 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:30.974037 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74027381-c8f5-440b-b9ad-13ef1757cdae None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:31.090430 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74027381-c8f5-440b-b9ad-13ef1757cdae None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:31.118840 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:31.127919 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105d5f94-8946-4373-b159-c2a76a1c14d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:31.128532 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d5f94-8946-4373-b159-c2a76a1c14d8 None admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:31.128812 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d5f94-8946-4373-b159-c2a76a1c14d8 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:31.129083 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d5f94-8946-4373-b159-c2a76a1c14d8 None admin] PATH_INFO: `/v3/services` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:31.129729 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105d5f94-8946-4373-b159-c2a76a1c14d8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:31.131812 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105d5f94-8946-4373-b159-c2a76a1c14d8 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:31.149764 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 437/873] 10.222.0.15 () {60 vars in 1141 bytes} [Sun Sep 11 10:34:30 2022] POST /identity/v3/services => generated 251 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:34:32.300087 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc29f8-55ed-42ac-9505-d8eb76bb58ac None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:32.300633 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc29f8-55ed-42ac-9505-d8eb76bb58ac None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:32.300633 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc29f8-55ed-42ac-9505-d8eb76bb58ac None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:32.302187 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 437/874] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:34:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:34:32.308270 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c264c-45e1-436e-83e2-00f418946c25 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:32.308697 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c264c-45e1-436e-83e2-00f418946c25 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:32.308900 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c264c-45e1-436e-83e2-00f418946c25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:32.329728 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-c67c264c-45e1-436e-83e2-00f418946c25 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:34:32.330919 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-c67c264c-45e1-436e-83e2-00f418946c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:34:32.405960 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 438/875] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:34:32 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 10:34:32.675155 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acc29f8-55ed-42ac-9505-d8eb76bb58ac None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:32.680691 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:32.783701 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acc29f8-55ed-42ac-9505-d8eb76bb58ac None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:32.821166 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76bbb48-eaa9-4e48-8936-0714ba6a4e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:32.822151 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76bbb48-eaa9-4e48-8936-0714ba6a4e0e None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:32.822629 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76bbb48-eaa9-4e48-8936-0714ba6a4e0e None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:32.823139 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76bbb48-eaa9-4e48-8936-0714ba6a4e0e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:32.823940 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76bbb48-eaa9-4e48-8936-0714ba6a4e0e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:32.825520 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76bbb48-eaa9-4e48-8936-0714ba6a4e0e None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:32.836244 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-b76bbb48-eaa9-4e48-8936-0714ba6a4e0e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 10:34:32.836244 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:32.836244 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 10:34:32.836244 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 10:34:32.836244 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 10:34:32.836244 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 10:34:32.836244 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 10:34:32.836244 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 10:34:32.836244 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 10:34:32.836244 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:32.836244 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 10:34:32.836244 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:32.836244 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:32.836244 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:32.836244 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:32.836244 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:32.836244 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:32.837328 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:32.837328 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:32.837328 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:32.837328 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:32.837328 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:32.837328 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:32.837328 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 10:34:32.837328 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 10:34:32.837328 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 10:34:32.837328 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 10:34:32.837328 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:32.837328 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:32.837328 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:32.837328 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:32.837328 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:32.837328 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:32.838166 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:32.838166 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:32.838166 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:32.838166 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:32.838166 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:32.838166 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:32.838166 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:32.838166 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:32.838166 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:32.838166 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:32.838166 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 10:34:32.838166 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 10:34:32.838166 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 10:34:32.838166 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:32.841277 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 438/876] 10.222.0.15 () {56 vars in 1126 bytes} [Sun Sep 11 10:34:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:32.850853 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c264c-45e1-436e-83e2-00f418946c25 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:32.953691 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c264c-45e1-436e-83e2-00f418946c25 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:32.965330 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913e051-53c2-4017-a8f3-e93dec7ff2d3 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:32.965915 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913e051-53c2-4017-a8f3-e93dec7ff2d3 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:32.966399 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913e051-53c2-4017-a8f3-e93dec7ff2d3 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:32.969187 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:32.969456 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 439/877] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:34:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 10:34:32.973838 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c85456f-3a41-4b90-affa-2e32aea4a182 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:32.974384 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c85456f-3a41-4b90-affa-2e32aea4a182 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:32.975505 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c85456f-3a41-4b90-affa-2e32aea4a182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:32.990200 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74b8f30-7c4d-42b8-898e-73141114d4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:32.990895 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74b8f30-7c4d-42b8-898e-73141114d4d2 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:32.991333 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74b8f30-7c4d-42b8-898e-73141114d4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:32.991550 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74b8f30-7c4d-42b8-898e-73141114d4d2 None admin] PATH_INFO: `/v3/services` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:32.992073 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74b8f30-7c4d-42b8-898e-73141114d4d2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:32.992354 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74b8f30-7c4d-42b8-898e-73141114d4d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:32.993727 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74b8f30-7c4d-42b8-898e-73141114d4d2 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:32.996609 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-0c85456f-3a41-4b90-affa-2e32aea4a182 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:34:32.998240 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-0c85456f-3a41-4b90-affa-2e32aea4a182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:34:32.999239 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 439/878] 10.222.0.15 () {56 vars in 1122 bytes} [Sun Sep 11 10:34:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:33.005009 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74b8f30-7c4d-42b8-898e-73141114d4d2 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:33.078810 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:33.095857 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 440/879] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:34:32 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 10:34:33.121741 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74b8f30-7c4d-42b8-898e-73141114d4d2 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:33.149676 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7642c72a-b5d1-4722-a65d-fafddaf453bf None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:33.149861 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7642c72a-b5d1-4722-a65d-fafddaf453bf None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:33.150137 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7642c72a-b5d1-4722-a65d-fafddaf453bf None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:33.151767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 441/880] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:34:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 10:34:33.156581 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6104e2b-66d7-4039-9845-ca13920f7a40 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:33.156906 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6104e2b-66d7-4039-9845-ca13920f7a40 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:33.157113 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6104e2b-66d7-4039-9845-ca13920f7a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:33.163594 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc7a212-1cbc-4a65-86d3-e40ae19334be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:33.164663 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc7a212-1cbc-4a65-86d3-e40ae19334be None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:33.165360 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc7a212-1cbc-4a65-86d3-e40ae19334be None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:33.166647 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc7a212-1cbc-4a65-86d3-e40ae19334be None admin] PATH_INFO: `/v3/services` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:33.167536 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc7a212-1cbc-4a65-86d3-e40ae19334be None admin] RBAC: Adding query filter params (type=object-store) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:33.168082 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc7a212-1cbc-4a65-86d3-e40ae19334be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:33.169694 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc7a212-1cbc-4a65-86d3-e40ae19334be None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:33.175726 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 440/881] 10.222.0.15 () {56 vars in 1122 bytes} [Sun Sep 11 10:34:33 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:34:33.176778 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-f6104e2b-66d7-4039-9845-ca13920f7a40 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:34:33.178095 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-f6104e2b-66d7-4039-9845-ca13920f7a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:34:33.263792 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 442/882] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:34:33 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 10:34:33.370677 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614d1ed1-7b60-4df8-9fc6-4b2c9fe6c326 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:33.371208 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614d1ed1-7b60-4df8-9fc6-4b2c9fe6c326 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:33.371208 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614d1ed1-7b60-4df8-9fc6-4b2c9fe6c326 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:33.372827 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 441/883] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:34:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:34:33.377854 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f6d60-f0b1-48e3-885d-c3bcb95886c3 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:33.378255 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f6d60-f0b1-48e3-885d-c3bcb95886c3 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:33.378413 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f6d60-f0b1-48e3-885d-c3bcb95886c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:33.389143 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614d1ed1-7b60-4df8-9fc6-4b2c9fe6c326 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:33.397852 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-e41f6d60-f0b1-48e3-885d-c3bcb95886c3 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:34:33.399037 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-e41f6d60-f0b1-48e3-885d-c3bcb95886c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:34:33.444487 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:33.476837 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 443/884] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:34:33 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 10:34:33.508603 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614d1ed1-7b60-4df8-9fc6-4b2c9fe6c326 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:33.544059 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407fedd0-a307-4936-a683-4846929f8e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:33.544691 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407fedd0-a307-4936-a683-4846929f8e9c None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:33.544963 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407fedd0-a307-4936-a683-4846929f8e9c None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:33.545232 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407fedd0-a307-4936-a683-4846929f8e9c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:33.558146 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407fedd0-a307-4936-a683-4846929f8e9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:34:33.562920 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-407fedd0-a307-4936-a683-4846929f8e9c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 10:34:33.562920 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:33.562920 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:33.562920 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:33.562920 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:33.562920 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:33.562920 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:33.562920 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:33.562920 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:33.562920 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:33.562920 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:33.562920 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:33.562920 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:34:33.562920 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:34:33.562920 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:34:33.562920 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:34:33.563903 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:34:33.563903 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:34:33.563903 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:34:33.563903 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:34:33.563903 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:33.563903 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:33.563903 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:33.563903 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:33.563903 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:33.563903 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:33.563903 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:33.563903 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:33.563903 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:33.563903 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:33.563903 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:33.563903 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:33.564749 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:33.564749 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:33.564749 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:33.564749 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:33.564749 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:34:33.564749 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:34:33.564749 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:34:33.564749 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:34:33.564749 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:34:33.564749 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:34:33.564749 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 10:34:33.564749 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:33.569092 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 442/885] 10.222.0.15 () {56 vars in 1120 bytes} [Sun Sep 11 10:34:33 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 10:34:33.570850 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41f6d60-f0b1-48e3-885d-c3bcb95886c3 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:33.576397 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407fedd0-a307-4936-a683-4846929f8e9c None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:33.649986 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:33.700068 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407fedd0-a307-4936-a683-4846929f8e9c None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:33.702364 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:33.707678 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41f6d60-f0b1-48e3-885d-c3bcb95886c3 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:33.745309 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636d1446-b2ee-429c-883f-ce0c488625f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:33.746480 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636d1446-b2ee-429c-883f-ce0c488625f0 None admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:33.746956 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636d1446-b2ee-429c-883f-ce0c488625f0 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:33.747441 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636d1446-b2ee-429c-883f-ce0c488625f0 None admin] PATH_INFO: `/v3/services` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:33.748279 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636d1446-b2ee-429c-883f-ce0c488625f0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:33.749950 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636d1446-b2ee-429c-883f-ce0c488625f0 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:33.752937 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1143d53-79f7-4c2b-a2d6-805d1f24ef81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:33.753560 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1143d53-79f7-4c2b-a2d6-805d1f24ef81 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:33.754116 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1143d53-79f7-4c2b-a2d6-805d1f24ef81 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:33.754388 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1143d53-79f7-4c2b-a2d6-805d1f24ef81 None admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:33.755600 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1143d53-79f7-4c2b-a2d6-805d1f24ef81 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:33.755831 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1143d53-79f7-4c2b-a2d6-805d1f24ef81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:33.758629 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1143d53-79f7-4c2b-a2d6-805d1f24ef81 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:33.767642 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 444/886] 10.222.0.15 () {56 vars in 1115 bytes} [Sun Sep 11 10:34:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:33.771578 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1143d53-79f7-4c2b-a2d6-805d1f24ef81 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:33.775771 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 443/887] 10.222.0.15 () {60 vars in 1141 bytes} [Sun Sep 11 10:34:33 2022] POST /identity/v3/services => generated 269 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:34:33.777216 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636d1446-b2ee-429c-883f-ce0c488625f0 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:33.899171 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1143d53-79f7-4c2b-a2d6-805d1f24ef81 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:33.924417 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:33.932644 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636d1446-b2ee-429c-883f-ce0c488625f0 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:33.941432 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73236819-1ed9-4bb3-8bc3-d9f563a1e9c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:33.942420 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73236819-1ed9-4bb3-8bc3-d9f563a1e9c1 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:33.942883 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73236819-1ed9-4bb3-8bc3-d9f563a1e9c1 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:33.943328 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73236819-1ed9-4bb3-8bc3-d9f563a1e9c1 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:33.945699 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73236819-1ed9-4bb3-8bc3-d9f563a1e9c1 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:33.947341 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73236819-1ed9-4bb3-8bc3-d9f563a1e9c1 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:33.960168 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-73236819-1ed9-4bb3-8bc3-d9f563a1e9c1 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 11 10:34:33.960168 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:33.960168 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 10:34:33.960168 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 10:34:33.960168 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 10:34:33.960168 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 10:34:33.960168 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 10:34:33.960168 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 10:34:33.960168 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 11 10:34:33.960168 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:33.960168 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 10:34:33.960168 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:33.960168 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:33.960168 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:33.960168 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:33.960168 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:33.960168 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:33.961425 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:33.961425 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:33.961425 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:33.961425 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:33.961425 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:33.961425 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:33.961425 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 10:34:33.961425 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 10:34:33.961425 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 10:34:33.961425 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 10:34:33.961425 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:33.961425 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:33.961425 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:33.961425 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:33.961425 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:33.961425 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:33.962751 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:33.962751 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:33.962751 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:33.962751 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:33.962751 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:33.962751 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:33.962751 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:33.962751 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:33.962751 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:33.962751 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:33.962751 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 10:34:33.962751 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 10:34:33.962751 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 11 10:34:33.962751 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:33.962751 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 445/888] 10.222.0.15 () {56 vars in 1114 bytes} [Sun Sep 11 10:34:33 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:33.968008 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73236819-1ed9-4bb3-8bc3-d9f563a1e9c1 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:33.984587 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ecd3cd-3cb9-4731-8fbf-4ea0362ced38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:33.985334 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ecd3cd-3cb9-4731-8fbf-4ea0362ced38 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:33.985603 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ecd3cd-3cb9-4731-8fbf-4ea0362ced38 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:33.986384 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ecd3cd-3cb9-4731-8fbf-4ea0362ced38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:34.000831 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ecd3cd-3cb9-4731-8fbf-4ea0362ced38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:34.002338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ecd3cd-3cb9-4731-8fbf-4ea0362ced38 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:34.021777 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-73ecd3cd-3cb9-4731-8fbf-4ea0362ced38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:34.021777 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:34.021777 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:34.021777 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:34.021777 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:34.021777 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:34.021777 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:34.021777 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:34.021777 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:34.021777 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:34.021777 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:34.021777 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:34.021777 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:34:34.021777 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:34:34.021777 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:34:34.021777 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:34:34.022826 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:34.022826 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:34.022826 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:34.022826 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:34.022826 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:34.022826 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:34.022826 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:34.022826 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:34.022826 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:34.022826 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:34.022826 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:34.022826 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:34.022826 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:34.022826 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:34.022826 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:34.022826 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:34.022826 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:34:34.023742 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:34:34.023742 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:34.023742 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:34.023742 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 444/889] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:34:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:34.028940 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ecd3cd-3cb9-4731-8fbf-4ea0362ced38 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:34.042310 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:34.095513 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73236819-1ed9-4bb3-8bc3-d9f563a1e9c1 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:34.134742 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b8b36b-bae2-42a4-b14a-c3ca18558db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:34.135361 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b8b36b-bae2-42a4-b14a-c3ca18558db3 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:34.135671 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b8b36b-bae2-42a4-b14a-c3ca18558db3 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:34.135946 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b8b36b-bae2-42a4-b14a-c3ca18558db3 None admin] PATH_INFO: `/v3/services` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:34.136571 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b8b36b-bae2-42a4-b14a-c3ca18558db3 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:34.136875 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b8b36b-bae2-42a4-b14a-c3ca18558db3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:34.138294 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b8b36b-bae2-42a4-b14a-c3ca18558db3 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:34.149291 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 446/890] 10.222.0.15 () {56 vars in 1114 bytes} [Sun Sep 11 10:34:33 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:34.150220 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b8b36b-bae2-42a4-b14a-c3ca18558db3 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:34.170069 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ecd3cd-3cb9-4731-8fbf-4ea0362ced38 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:34.205697 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:34.218650 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5661ec-2b97-4c71-a14e-97df17b1920d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:34.220002 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5661ec-2b97-4c71-a14e-97df17b1920d None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:34.220708 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5661ec-2b97-4c71-a14e-97df17b1920d None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:34.221163 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5661ec-2b97-4c71-a14e-97df17b1920d None admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:34.222268 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5661ec-2b97-4c71-a14e-97df17b1920d None admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:34.223083 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5661ec-2b97-4c71-a14e-97df17b1920d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:34.225032 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5661ec-2b97-4c71-a14e-97df17b1920d None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:34.246384 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 445/891] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:34:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:34.247200 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5661ec-2b97-4c71-a14e-97df17b1920d None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:34.277889 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b8b36b-bae2-42a4-b14a-c3ca18558db3 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:34.330364 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834c07c8-b8e2-4770-a07a-d5f5e10270eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:34.331331 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834c07c8-b8e2-4770-a07a-d5f5e10270eb None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:34.331923 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834c07c8-b8e2-4770-a07a-d5f5e10270eb None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:34.332648 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834c07c8-b8e2-4770-a07a-d5f5e10270eb None admin] PATH_INFO: `/v3/services` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:34.334645 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834c07c8-b8e2-4770-a07a-d5f5e10270eb None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:34.335291 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834c07c8-b8e2-4770-a07a-d5f5e10270eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:34.337358 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834c07c8-b8e2-4770-a07a-d5f5e10270eb None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:34.338103 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:34.352691 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 447/892] 10.222.0.15 () {56 vars in 1114 bytes} [Sun Sep 11 10:34:34 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:34.353551 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834c07c8-b8e2-4770-a07a-d5f5e10270eb None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:34.384985 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5661ec-2b97-4c71-a14e-97df17b1920d None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:34.431840 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9649d2b0-a37a-43f1-8066-5d729a602c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:34.432816 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9649d2b0-a37a-43f1-8066-5d729a602c95 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:34.433300 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9649d2b0-a37a-43f1-8066-5d729a602c95 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:34.433770 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9649d2b0-a37a-43f1-8066-5d729a602c95 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:34.442456 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9649d2b0-a37a-43f1-8066-5d729a602c95 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:34.444930 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9649d2b0-a37a-43f1-8066-5d729a602c95 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:34.458134 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-9649d2b0-a37a-43f1-8066-5d729a602c95 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 10:34:34.458134 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:34.458134 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:34.458134 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:34.458134 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:34.458134 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:34.458134 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:34.458134 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:34.458134 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:34.458134 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:34.458134 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:34.458134 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:34.458134 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 10:34:34.458134 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 10:34:34.458134 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 10:34:34.458134 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 10:34:34.459372 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:34.459372 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:34.459372 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 10:34:34.459372 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:34:34.459372 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 10:34:34.459372 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:34:34.459372 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:34.459372 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:34.459372 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:34.459372 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:34.459372 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:34.459372 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:34.459372 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:34.459372 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:34.459372 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:34.459372 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:34.459372 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:34.459372 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:34.460380 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:34.460380 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:34.460380 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 10:34:34.460380 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 10:34:34.460380 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 10:34:34.460380 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 10:34:34.460380 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 10:34:34.460380 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 10:34:34.460380 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 10:34:34.460380 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:34.462912 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 446/893] 10.222.0.15 () {56 vars in 1117 bytes} [Sun Sep 11 10:34:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 10:34:34.468712 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9649d2b0-a37a-43f1-8066-5d729a602c95 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:34.477966 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834c07c8-b8e2-4770-a07a-d5f5e10270eb None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:34.493257 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:34.517944 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7178799-e5c0-4018-8fef-b0074650bb25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:34.518905 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7178799-e5c0-4018-8fef-b0074650bb25 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:34.519380 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7178799-e5c0-4018-8fef-b0074650bb25 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:34.520008 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7178799-e5c0-4018-8fef-b0074650bb25 None admin] PATH_INFO: `/v3/endpoints` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:34.520898 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7178799-e5c0-4018-8fef-b0074650bb25 None admin] RBAC: Adding query filter params (interface=public, service_id=02b81d1a06da427ab3edd261bd28c424, region_id=RegionOne) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:34.521521 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7178799-e5c0-4018-8fef-b0074650bb25 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:34.523380 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7178799-e5c0-4018-8fef-b0074650bb25 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:34.529565 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 448/894] 10.222.0.15 () {56 vars in 1251 bytes} [Sun Sep 11 10:34:34 2022] GET /identity/v3/endpoints?service_id=02b81d1a06da427ab3edd261bd28c424&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:34.534655 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7178799-e5c0-4018-8fef-b0074650bb25 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:34.583053 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9649d2b0-a37a-43f1-8066-5d729a602c95 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:34.614570 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:34.621368 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd1eac7-1d6f-4b2f-824e-fe2b1d4f715d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:34.621979 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd1eac7-1d6f-4b2f-824e-fe2b1d4f715d None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:34.622256 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd1eac7-1d6f-4b2f-824e-fe2b1d4f715d None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:34.622523 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd1eac7-1d6f-4b2f-824e-fe2b1d4f715d None admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:34.623242 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd1eac7-1d6f-4b2f-824e-fe2b1d4f715d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:34.623828 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd1eac7-1d6f-4b2f-824e-fe2b1d4f715d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:34.625097 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd1eac7-1d6f-4b2f-824e-fe2b1d4f715d None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:34.647456 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7178799-e5c0-4018-8fef-b0074650bb25 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:34.649637 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 447/895] 10.222.0.15 () {56 vars in 1149 bytes} [Sun Sep 11 10:34:34 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:34:34.650549 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd1eac7-1d6f-4b2f-824e-fe2b1d4f715d None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:34.686909 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a8c1a1-434d-4b55-b696-a7f9a4740197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:34.687860 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8c1a1-434d-4b55-b696-a7f9a4740197 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:34.688360 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8c1a1-434d-4b55-b696-a7f9a4740197 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:34.688874 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8c1a1-434d-4b55-b696-a7f9a4740197 None admin] PATH_INFO: `/v3/services` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:34.689775 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a8c1a1-434d-4b55-b696-a7f9a4740197 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:34.691379 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a8c1a1-434d-4b55-b696-a7f9a4740197 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:34.698239 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 449/896] 10.222.0.15 () {56 vars in 1087 bytes} [Sun Sep 11 10:34:34 2022] GET /identity/v3/services => generated 1268 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:34:34.767332 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd1eac7-1d6f-4b2f-824e-fe2b1d4f715d None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:34.806720 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029b00d1-acdc-4224-94e2-a226cf2b71ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:34.807327 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b00d1-acdc-4224-94e2-a226cf2b71ef None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:34.807630 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b00d1-acdc-4224-94e2-a226cf2b71ef None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:34.807899 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b00d1-acdc-4224-94e2-a226cf2b71ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:34.871930 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:34.884598 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029b00d1-acdc-4224-94e2-a226cf2b71ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:34.886069 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029b00d1-acdc-4224-94e2-a226cf2b71ef None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:34.990933 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 448/897] 10.222.0.15 () {58 vars in 1282 bytes} [Sun Sep 11 10:34:34 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 10:34:34.996960 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a8c1a1-434d-4b55-b696-a7f9a4740197 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:35.040279 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:35.107513 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a8c1a1-434d-4b55-b696-a7f9a4740197 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:35.140792 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb6fbf-468c-45b3-8959-307c6295924a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:35.141435 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb6fbf-468c-45b3-8959-307c6295924a None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:35.141711 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb6fbf-468c-45b3-8959-307c6295924a None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:35.141979 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb6fbf-468c-45b3-8959-307c6295924a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:35.154122 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cb6fbf-468c-45b3-8959-307c6295924a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:35.155650 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cb6fbf-468c-45b3-8959-307c6295924a None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:35.170191 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-78cb6fbf-468c-45b3-8959-307c6295924a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:35.170191 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:35.170191 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:35.170191 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:35.170191 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:35.170191 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:35.170191 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:35.170191 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:35.170191 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:35.170191 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:35.170191 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:35.170191 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:35.170191 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:34:35.170191 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:34:35.170191 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:34:35.170191 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:34:35.171167 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:35.171167 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:35.171167 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:35.171167 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:35.171167 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:35.171167 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:35.171167 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:35.171167 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:35.171167 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:35.171167 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:35.171167 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:35.171167 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:35.171167 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:35.171167 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:35.171167 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:35.171167 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:35.171167 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:34:35.172100 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:34:35.172100 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:35.172100 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:35.172100 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 450/898] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:34:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:35.179125 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029b00d1-acdc-4224-94e2-a226cf2b71ef None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:35.276909 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:35.293288 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029b00d1-acdc-4224-94e2-a226cf2b71ef None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:35.332458 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b22daf-61c2-4687-a490-646159591995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:35.333411 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b22daf-61c2-4687-a490-646159591995 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:35.333893 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b22daf-61c2-4687-a490-646159591995 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:35.334425 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b22daf-61c2-4687-a490-646159591995 None admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:35.335350 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b22daf-61c2-4687-a490-646159591995 None admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:35.335912 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b22daf-61c2-4687-a490-646159591995 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:35.337483 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b22daf-61c2-4687-a490-646159591995 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:35.350557 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 449/899] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:34:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:35.356285 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb6fbf-468c-45b3-8959-307c6295924a None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:35.386418 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7542f4ee-7e8a-4287-af52-ebba9380bd66 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:35.386932 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7542f4ee-7e8a-4287-af52-ebba9380bd66 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:35.387048 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7542f4ee-7e8a-4287-af52-ebba9380bd66 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:35.388551 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 450/900] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:34:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 10:34:35.393392 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b86a418-dc7f-435f-b867-7553ff2f6da3 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:35.393676 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b86a418-dc7f-435f-b867-7553ff2f6da3 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:35.393955 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b86a418-dc7f-435f-b867-7553ff2f6da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:35.413161 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-9b86a418-dc7f-435f-b867-7553ff2f6da3 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:34:35.414344 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-9b86a418-dc7f-435f-b867-7553ff2f6da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:34:35.457853 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:35.470223 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb6fbf-468c-45b3-8959-307c6295924a None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:35.496238 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 451/901] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:34:35 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 10:34:35.516664 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9129b2-f4f9-4089-b2f3-9bd9ce2615e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:35.518022 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9129b2-f4f9-4089-b2f3-9bd9ce2615e7 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:35.518505 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9129b2-f4f9-4089-b2f3-9bd9ce2615e7 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:35.519004 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9129b2-f4f9-4089-b2f3-9bd9ce2615e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:35.527784 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9129b2-f4f9-4089-b2f3-9bd9ce2615e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:35.529513 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9129b2-f4f9-4089-b2f3-9bd9ce2615e7 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:35.542132 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-fe9129b2-f4f9-4089-b2f3-9bd9ce2615e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:34:35.542132 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:35.542132 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:35.542132 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:35.542132 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:35.542132 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:35.542132 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:35.542132 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:35.542132 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:35.542132 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:35.542132 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:35.542132 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:35.542132 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 10:34:35.542132 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 10:34:35.542132 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 10:34:35.543050 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 10:34:35.543050 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:35.543050 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:35.543050 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:35.543050 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:35.543050 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:35.543050 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:35.543050 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:35.543050 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:35.543050 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:35.543050 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:35.543050 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:35.543050 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:35.543050 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:35.543050 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:35.543050 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:35.543050 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:35.543988 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 10:34:35.543988 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 10:34:35.543988 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 10:34:35.543988 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 10:34:35.543988 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 10:34:35.543988 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 10:34:35.543988 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:34:35.543988 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:35.550823 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 451/902] 10.222.0.15 () {56 vars in 1111 bytes} [Sun Sep 11 10:34:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:35.552390 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b86a418-dc7f-435f-b867-7553ff2f6da3 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:35.640386 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:35.663938 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b86a418-dc7f-435f-b867-7553ff2f6da3 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:35.702602 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143a2916-0e1d-470a-9bba-b6091787a064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:35.703223 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143a2916-0e1d-470a-9bba-b6091787a064 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:35.703532 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143a2916-0e1d-470a-9bba-b6091787a064 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:35.703809 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143a2916-0e1d-470a-9bba-b6091787a064 None admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:35.704508 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143a2916-0e1d-470a-9bba-b6091787a064 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:35.704810 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143a2916-0e1d-470a-9bba-b6091787a064 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:35.706235 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143a2916-0e1d-470a-9bba-b6091787a064 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:35.718928 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 452/903] 10.222.0.15 () {56 vars in 1148 bytes} [Sun Sep 11 10:34:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:35.725584 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9129b2-f4f9-4089-b2f3-9bd9ce2615e7 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:35.774278 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143a2916-0e1d-470a-9bba-b6091787a064 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:35.838955 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9129b2-f4f9-4089-b2f3-9bd9ce2615e7 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:35.876775 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a7c830-3e01-4a41-b4ac-ff5087a901a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:35.877442 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a7c830-3e01-4a41-b4ac-ff5087a901a6 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:35.877726 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a7c830-3e01-4a41-b4ac-ff5087a901a6 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:35.878003 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a7c830-3e01-4a41-b4ac-ff5087a901a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:35.878543 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143a2916-0e1d-470a-9bba-b6091787a064 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:35.878775 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a7c830-3e01-4a41-b4ac-ff5087a901a6 None admin] RBAC: Adding query filter params (role.id=55b167894e584c9b8303c60cf3a75a65, scope.project.id=5e1f845f99f04202b83621cb65e0adc2, user.id=bb44e7aef0c7436d93a7233307b5f095) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:35.879088 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a7c830-3e01-4a41-b4ac-ff5087a901a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:35.880663 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a7c830-3e01-4a41-b4ac-ff5087a901a6 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:35.895618 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 452/904] 10.222.0.15 () {56 vars in 1374 bytes} [Sun Sep 11 10:34:35 2022] GET /identity/v3/role_assignments?user.id=bb44e7aef0c7436d93a7233307b5f095&scope.project.id=5e1f845f99f04202b83621cb65e0adc2&role.id=55b167894e584c9b8303c60cf3a75a65 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:35.917067 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31236622-9cba-4ad6-987a-5c5db86ca6fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:35.918108 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31236622-9cba-4ad6-987a-5c5db86ca6fe None admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:35.918589 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31236622-9cba-4ad6-987a-5c5db86ca6fe None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:35.919055 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31236622-9cba-4ad6-987a-5c5db86ca6fe None admin] PATH_INFO: `/v3/services` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:35.920058 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31236622-9cba-4ad6-987a-5c5db86ca6fe None admin] RBAC: Authorizing `identity:create_service()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:35.921695 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31236622-9cba-4ad6-987a-5c5db86ca6fe None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:35.934027 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:35.940931 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 453/905] 10.222.0.15 () {60 vars in 1141 bytes} [Sun Sep 11 10:34:35 2022] POST /identity/v3/services => generated 241 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 10:34:36.173334 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b71b77-17c7-4446-88c7-bfaccd94782a None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:36.173629 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b71b77-17c7-4446-88c7-bfaccd94782a None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:36.173802 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b71b77-17c7-4446-88c7-bfaccd94782a None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:36.175155 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 453/906] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:34:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:34:36.180576 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dad5ced-e81a-4f2d-ba4e-5128f1d830e9 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:36.180934 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dad5ced-e81a-4f2d-ba4e-5128f1d830e9 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:36.181283 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dad5ced-e81a-4f2d-ba4e-5128f1d830e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:36.202039 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-5dad5ced-e81a-4f2d-ba4e-5128f1d830e9 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:34:36.203280 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-5dad5ced-e81a-4f2d-ba4e-5128f1d830e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:34:36.278974 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 454/907] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:34:36 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 10:34:36.551808 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b71b77-17c7-4446-88c7-bfaccd94782a None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:36.573934 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:36.668425 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b71b77-17c7-4446-88c7-bfaccd94782a None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:36.705474 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea13d5b-17e6-4acc-a4e1-5f05b90b27d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:36.706111 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea13d5b-17e6-4acc-a4e1-5f05b90b27d0 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:36.706392 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea13d5b-17e6-4acc-a4e1-5f05b90b27d0 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:36.706665 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea13d5b-17e6-4acc-a4e1-5f05b90b27d0 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:36.707243 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea13d5b-17e6-4acc-a4e1-5f05b90b27d0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:36.708655 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea13d5b-17e6-4acc-a4e1-5f05b90b27d0 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:36.718388 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-2ea13d5b-17e6-4acc-a4e1-5f05b90b27d0 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 11 10:34:36.718388 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:36.718388 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 10:34:36.718388 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 10:34:36.718388 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 10:34:36.718388 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 10:34:36.718388 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 10:34:36.718388 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 10:34:36.718388 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 11 10:34:36.718388 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:36.718388 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 10:34:36.718388 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:36.718388 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:36.718388 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:36.718388 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:36.718388 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:36.719368 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:36.719368 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:36.719368 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:36.719368 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:36.719368 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:36.719368 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:36.719368 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:36.719368 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 10:34:36.719368 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 10:34:36.719368 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 10:34:36.719368 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 10:34:36.719368 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:36.719368 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:36.719368 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:36.719368 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:36.719368 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:36.719368 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:36.720353 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:36.720353 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:36.720353 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:36.720353 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:36.720353 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:36.720353 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:36.720353 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:36.720353 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:36.720353 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:36.720353 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:36.720353 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 10:34:36.720353 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 10:34:36.720353 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 11 10:34:36.720353 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:36.720353 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 454/908] 10.222.0.15 () {56 vars in 1132 bytes} [Sun Sep 11 10:34:36 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:36.727654 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dad5ced-e81a-4f2d-ba4e-5128f1d830e9 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:36.836156 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dad5ced-e81a-4f2d-ba4e-5128f1d830e9 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:36.863532 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:36.876834 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3f7a28-46c1-4df4-bea5-cc5950531251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:36.877467 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f7a28-46c1-4df4-bea5-cc5950531251 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:36.877745 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f7a28-46c1-4df4-bea5-cc5950531251 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:36.878016 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f7a28-46c1-4df4-bea5-cc5950531251 None admin] PATH_INFO: `/v3/services` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:36.878672 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3f7a28-46c1-4df4-bea5-cc5950531251 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:36.878974 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3f7a28-46c1-4df4-bea5-cc5950531251 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:36.880483 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3f7a28-46c1-4df4-bea5-cc5950531251 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:36.887767 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 455/909] 10.222.0.15 () {56 vars in 1126 bytes} [Sun Sep 11 10:34:36 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:36.893582 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea13d5b-17e6-4acc-a4e1-5f05b90b27d0 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:36.981179 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:37.002883 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea13d5b-17e6-4acc-a4e1-5f05b90b27d0 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:37.009956 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c872fa43-ec30-4bdc-931a-a191044e8a6e None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:37.010241 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c872fa43-ec30-4bdc-931a-a191044e8a6e None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:37.010507 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c872fa43-ec30-4bdc-931a-a191044e8a6e None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:37.012010 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 456/910] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:34:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 10:34:37.019822 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3cbc95-1872-4e8a-82ad-e4d7a76723e7 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:37.019822 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3cbc95-1872-4e8a-82ad-e4d7a76723e7 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:37.019822 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3cbc95-1872-4e8a-82ad-e4d7a76723e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:37.038736 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-8b3cbc95-1872-4e8a-82ad-e4d7a76723e7 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:34:37.040047 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-8b3cbc95-1872-4e8a-82ad-e4d7a76723e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:34:37.042657 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4206ddb2-381f-47f9-8d92-c91927887ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:37.043481 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4206ddb2-381f-47f9-8d92-c91927887ef2 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:37.043992 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4206ddb2-381f-47f9-8d92-c91927887ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:37.044410 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4206ddb2-381f-47f9-8d92-c91927887ef2 None admin] PATH_INFO: `/v3/services` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:37.045194 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4206ddb2-381f-47f9-8d92-c91927887ef2 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:37.045642 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4206ddb2-381f-47f9-8d92-c91927887ef2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:37.047147 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4206ddb2-381f-47f9-8d92-c91927887ef2 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:37.053527 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 455/911] 10.222.0.15 () {56 vars in 1126 bytes} [Sun Sep 11 10:34:36 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:37.063474 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4206ddb2-381f-47f9-8d92-c91927887ef2 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:37.119984 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 457/912] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:34:37 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 10:34:37.185431 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4206ddb2-381f-47f9-8d92-c91927887ef2 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:37.220990 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eac34f3-dc0e-41c2-aee7-d11616372f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:37.221928 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eac34f3-dc0e-41c2-aee7-d11616372f80 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:37.222351 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eac34f3-dc0e-41c2-aee7-d11616372f80 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:37.222766 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eac34f3-dc0e-41c2-aee7-d11616372f80 None admin] PATH_INFO: `/v3/endpoints` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:37.223639 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eac34f3-dc0e-41c2-aee7-d11616372f80 None admin] RBAC: Adding query filter params (interface=public, service_id=ff1256426c1245308f87ca6c39a56347, region_id=RegionOne) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:37.224107 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eac34f3-dc0e-41c2-aee7-d11616372f80 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:37.225781 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eac34f3-dc0e-41c2-aee7-d11616372f80 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:37.232033 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 456/913] 10.222.0.15 () {56 vars in 1251 bytes} [Sun Sep 11 10:34:37 2022] GET /identity/v3/endpoints?service_id=ff1256426c1245308f87ca6c39a56347&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:37.241147 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3cbc95-1872-4e8a-82ad-e4d7a76723e7 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:37.351822 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3cbc95-1872-4e8a-82ad-e4d7a76723e7 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:37.370151 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:37.387576 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c074947-e0ab-4e65-9e9f-65bd1c192a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:37.388182 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c074947-e0ab-4e65-9e9f-65bd1c192a32 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:37.388465 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c074947-e0ab-4e65-9e9f-65bd1c192a32 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:37.388737 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c074947-e0ab-4e65-9e9f-65bd1c192a32 None admin] PATH_INFO: `/v3/services` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:37.389337 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c074947-e0ab-4e65-9e9f-65bd1c192a32 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:37.390689 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c074947-e0ab-4e65-9e9f-65bd1c192a32 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:37.395392 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eac34f3-dc0e-41c2-aee7-d11616372f80 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:37.399076 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 458/914] 10.222.0.15 () {56 vars in 1087 bytes} [Sun Sep 11 10:34:37 2022] GET /identity/v3/services => generated 1497 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:34:37.408015 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:37.513387 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eac34f3-dc0e-41c2-aee7-d11616372f80 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:37.552404 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6abc25f-4e23-4153-b616-a6c3824a65df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:37.553686 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6abc25f-4e23-4153-b616-a6c3824a65df None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:37.553900 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6abc25f-4e23-4153-b616-a6c3824a65df None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:37.554395 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6abc25f-4e23-4153-b616-a6c3824a65df None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:37.554937 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6abc25f-4e23-4153-b616-a6c3824a65df None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:37.556368 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6abc25f-4e23-4153-b616-a6c3824a65df None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:37.566213 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-a6abc25f-4e23-4153-b616-a6c3824a65df None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 11 10:34:37.566213 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:37.566213 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 10:34:37.566213 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 10:34:37.566213 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 10:34:37.566213 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 10:34:37.566213 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 10:34:37.566213 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 10:34:37.566213 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 11 10:34:37.566213 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:37.566213 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 10:34:37.566213 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:37.566213 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:37.566213 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:37.566213 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:37.566213 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:37.566213 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:37.569189 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:37.569189 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:37.569189 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:37.569189 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:37.569189 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:37.569189 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:37.569189 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 10:34:37.569189 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 10:34:37.569189 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 10:34:37.569189 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 10:34:37.569189 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:37.569189 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:37.569189 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:37.569189 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:37.569189 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:37.569189 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:37.570303 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:37.570303 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:37.570303 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:37.570303 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:37.570303 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:37.570303 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:37.570303 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:37.570303 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:37.570303 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:37.570303 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:37.570303 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 10:34:37.570303 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 10:34:37.570303 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 11 10:34:37.570303 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:37.570303 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 457/915] 10.222.0.15 () {56 vars in 1114 bytes} [Sun Sep 11 10:34:37 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 10:34:37.574871 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c074947-e0ab-4e65-9e9f-65bd1c192a32 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:37.690881 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c074947-e0ab-4e65-9e9f-65bd1c192a32 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:37.728220 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232c2121-9944-493b-ae59-dfcd239f0b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:37.729619 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232c2121-9944-493b-ae59-dfcd239f0b24 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:37.730131 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232c2121-9944-493b-ae59-dfcd239f0b24 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:37.730627 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232c2121-9944-493b-ae59-dfcd239f0b24 None admin] PATH_INFO: `/v3/services` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:37.731477 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232c2121-9944-493b-ae59-dfcd239f0b24 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:37.731980 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232c2121-9944-493b-ae59-dfcd239f0b24 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:37.733528 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232c2121-9944-493b-ae59-dfcd239f0b24 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:37.739232 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 459/916] 10.222.0.15 () {56 vars in 1114 bytes} [Sun Sep 11 10:34:37 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:34:37.748358 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6abc25f-4e23-4153-b616-a6c3824a65df None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:37.820159 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:37.852773 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6abc25f-4e23-4153-b616-a6c3824a65df None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:37.887708 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275dc4b9-fbcc-4ae4-8cbb-0111b167445c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:37.888325 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275dc4b9-fbcc-4ae4-8cbb-0111b167445c None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:37.888660 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275dc4b9-fbcc-4ae4-8cbb-0111b167445c None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:37.888961 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275dc4b9-fbcc-4ae4-8cbb-0111b167445c None admin] PATH_INFO: `/v3/services` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:37.889610 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275dc4b9-fbcc-4ae4-8cbb-0111b167445c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:37.889921 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275dc4b9-fbcc-4ae4-8cbb-0111b167445c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:37.891294 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275dc4b9-fbcc-4ae4-8cbb-0111b167445c None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:37.897103 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 458/917] 10.222.0.15 () {56 vars in 1114 bytes} [Sun Sep 11 10:34:37 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:37.902882 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232c2121-9944-493b-ae59-dfcd239f0b24 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:37.954500 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:38.018842 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232c2121-9944-493b-ae59-dfcd239f0b24 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:38.056647 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc46e3e2-1357-4784-a669-16df12972d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:38.057286 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc46e3e2-1357-4784-a669-16df12972d4d None admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:38.057567 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc46e3e2-1357-4784-a669-16df12972d4d None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:38.057836 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc46e3e2-1357-4784-a669-16df12972d4d None admin] PATH_INFO: `/v3/endpoints` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:38.058542 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc46e3e2-1357-4784-a669-16df12972d4d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:38.060005 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc46e3e2-1357-4784-a669-16df12972d4d None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:38.087047 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 460/918] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:34:37 2022] POST /identity/v3/endpoints => generated 346 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:34:38.180353 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b073d6bd-5399-4f89-b056-8456e4a24053 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:38.180817 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b073d6bd-5399-4f89-b056-8456e4a24053 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:38.181201 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b073d6bd-5399-4f89-b056-8456e4a24053 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:38.183071 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 459/919] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:34:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:34:38.188714 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d94b721-615a-4949-a14b-2d2a0f4bc0de None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:38.189010 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d94b721-615a-4949-a14b-2d2a0f4bc0de None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:38.189272 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d94b721-615a-4949-a14b-2d2a0f4bc0de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:38.210206 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-4d94b721-615a-4949-a14b-2d2a0f4bc0de None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:34:38.211389 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-4d94b721-615a-4949-a14b-2d2a0f4bc0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:34:38.214227 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30226b58-55d3-42de-88aa-966ea73eee79 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:38.214511 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30226b58-55d3-42de-88aa-966ea73eee79 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:38.214765 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30226b58-55d3-42de-88aa-966ea73eee79 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:38.216470 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 460/920] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:34:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:34:38.223587 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51005f84-9f17-484d-935d-1f6373805d63 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:38.224208 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51005f84-9f17-484d-935d-1f6373805d63 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:38.225291 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51005f84-9f17-484d-935d-1f6373805d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:38.246299 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-51005f84-9f17-484d-935d-1f6373805d63 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:34:38.247672 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-51005f84-9f17-484d-935d-1f6373805d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:34:38.326757 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 461/921] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:34:38 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 10:34:38.343448 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 461/922] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:34:38 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 10:34:38.594612 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d94b721-615a-4949-a14b-2d2a0f4bc0de None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:38.616618 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51005f84-9f17-484d-935d-1f6373805d63 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:38.625147 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:38.705556 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:38.706278 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d94b721-615a-4949-a14b-2d2a0f4bc0de None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:38.745432 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d55f4d-0d03-446c-b43b-f4c7de27204f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:38.746087 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d55f4d-0d03-446c-b43b-f4c7de27204f None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:38.746343 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d55f4d-0d03-446c-b43b-f4c7de27204f None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:38.746613 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d55f4d-0d03-446c-b43b-f4c7de27204f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:38.747195 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d55f4d-0d03-446c-b43b-f4c7de27204f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:38.748596 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d55f4d-0d03-446c-b43b-f4c7de27204f None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:38.758384 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-55d55f4d-0d03-446c-b43b-f4c7de27204f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 10:34:38.758384 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:38.758384 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 10:34:38.758384 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 10:34:38.758384 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 10:34:38.758384 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 10:34:38.758384 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 10:34:38.758384 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 10:34:38.758384 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 10:34:38.758384 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:38.758384 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 10:34:38.758384 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:38.758384 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:38.758384 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:38.758384 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:38.758384 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:38.758384 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:38.759500 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:38.759500 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:38.759500 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:38.759500 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:38.759500 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:38.759500 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:38.759500 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 10:34:38.759500 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 10:34:38.759500 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 10:34:38.759500 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 10:34:38.759500 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:38.759500 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:38.759500 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:38.759500 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:38.759500 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:38.759500 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:38.760354 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:38.760354 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:38.760354 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:38.760354 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:38.760354 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:38.760354 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:38.760354 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:38.760354 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:38.760354 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:38.760354 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:38.760354 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 10:34:38.760354 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 10:34:38.760354 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 10:34:38.760354 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:38.761091 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51005f84-9f17-484d-935d-1f6373805d63 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:38.768147 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 462/923] 10.222.0.15 () {56 vars in 1126 bytes} [Sun Sep 11 10:34:38 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:38.769627 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d55f4d-0d03-446c-b43b-f4c7de27204f None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:38.802079 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655e8363-6f43-420a-8beb-4e675f1c7984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:38.803474 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e8363-6f43-420a-8beb-4e675f1c7984 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:38.803855 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e8363-6f43-420a-8beb-4e675f1c7984 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:38.804380 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e8363-6f43-420a-8beb-4e675f1c7984 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:38.818067 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655e8363-6f43-420a-8beb-4e675f1c7984 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:34:38.825093 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-655e8363-6f43-420a-8beb-4e675f1c7984 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 10:34:38.825093 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:38.825093 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:38.825093 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:38.825093 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:38.825093 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:38.825093 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:38.825093 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:38.825093 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:38.825093 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:38.825093 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:38.825093 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:38.825093 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:34:38.825093 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:34:38.825093 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:34:38.825093 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:34:38.826252 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:34:38.826252 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:34:38.826252 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:34:38.826252 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:34:38.826252 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:38.826252 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:38.826252 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:38.826252 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:38.826252 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:38.826252 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:38.826252 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:38.826252 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:38.826252 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:38.826252 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:38.826252 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:38.826252 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:38.827288 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:38.827288 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:38.827288 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:38.827288 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:38.827288 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:34:38.827288 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:34:38.827288 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:34:38.827288 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:34:38.827288 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:34:38.827288 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:34:38.827288 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 10:34:38.827288 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:38.829868 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 462/924] 10.222.0.15 () {56 vars in 1120 bytes} [Sun Sep 11 10:34:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 10:34:38.837355 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655e8363-6f43-420a-8beb-4e675f1c7984 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:38.885205 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:38.894526 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d55f4d-0d03-446c-b43b-f4c7de27204f None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:38.937448 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf6567e-753a-4e10-9180-1618788956ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:38.938398 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf6567e-753a-4e10-9180-1618788956ab None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:38.938876 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf6567e-753a-4e10-9180-1618788956ab None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:38.939331 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf6567e-753a-4e10-9180-1618788956ab None admin] PATH_INFO: `/v3/services` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:38.940424 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf6567e-753a-4e10-9180-1618788956ab None admin] RBAC: Adding query filter params (name=object-store) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:38.941274 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf6567e-753a-4e10-9180-1618788956ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:38.942825 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf6567e-753a-4e10-9180-1618788956ab None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:38.949731 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 463/925] 10.222.0.15 () {56 vars in 1122 bytes} [Sun Sep 11 10:34:38 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:38.954996 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf6567e-753a-4e10-9180-1618788956ab None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:38.963665 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655e8363-6f43-420a-8beb-4e675f1c7984 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:38.994298 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:39.000261 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1552b0-00d9-407a-b33d-7ccbf1aeb8f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:39.000967 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1552b0-00d9-407a-b33d-7ccbf1aeb8f9 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:39.001356 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1552b0-00d9-407a-b33d-7ccbf1aeb8f9 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:39.001708 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1552b0-00d9-407a-b33d-7ccbf1aeb8f9 None admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:39.002490 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1552b0-00d9-407a-b33d-7ccbf1aeb8f9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:39.002881 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1552b0-00d9-407a-b33d-7ccbf1aeb8f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:39.004388 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1552b0-00d9-407a-b33d-7ccbf1aeb8f9 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:39.014167 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 463/926] 10.222.0.15 () {56 vars in 1115 bytes} [Sun Sep 11 10:34:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:39.019651 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1552b0-00d9-407a-b33d-7ccbf1aeb8f9 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:39.078272 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf6567e-753a-4e10-9180-1618788956ab None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:39.118495 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52466ee8-1802-40d0-873b-efb718cf5c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:39.119775 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52466ee8-1802-40d0-873b-efb718cf5c5c None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:39.120239 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52466ee8-1802-40d0-873b-efb718cf5c5c None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:39.120688 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52466ee8-1802-40d0-873b-efb718cf5c5c None admin] PATH_INFO: `/v3/services` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:39.121495 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52466ee8-1802-40d0-873b-efb718cf5c5c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:39.121975 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52466ee8-1802-40d0-873b-efb718cf5c5c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:39.123554 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52466ee8-1802-40d0-873b-efb718cf5c5c None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:39.129922 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 464/927] 10.222.0.15 () {56 vars in 1122 bytes} [Sun Sep 11 10:34:38 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:39.137026 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52466ee8-1802-40d0-873b-efb718cf5c5c None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:39.139354 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1552b0-00d9-407a-b33d-7ccbf1aeb8f9 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:39.160462 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:39.189256 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a124d0d5-d1e1-4f7d-b67e-6fb80fcfb2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:39.189906 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a124d0d5-d1e1-4f7d-b67e-6fb80fcfb2a9 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:39.190242 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a124d0d5-d1e1-4f7d-b67e-6fb80fcfb2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:39.190537 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a124d0d5-d1e1-4f7d-b67e-6fb80fcfb2a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:39.205721 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a124d0d5-d1e1-4f7d-b67e-6fb80fcfb2a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:39.207490 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a124d0d5-d1e1-4f7d-b67e-6fb80fcfb2a9 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:39.226611 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-a124d0d5-d1e1-4f7d-b67e-6fb80fcfb2a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:39.226611 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:39.226611 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:39.226611 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:39.226611 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:39.226611 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:39.226611 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:39.226611 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:39.226611 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:39.226611 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:39.226611 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:39.226611 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:39.226611 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:34:39.226611 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:34:39.226611 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:34:39.226611 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:34:39.227996 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:39.227996 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:39.227996 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:39.227996 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:39.227996 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:39.227996 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:39.227996 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:39.227996 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:39.227996 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:39.227996 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:39.227996 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:39.227996 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:39.227996 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:39.227996 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:39.227996 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:39.227996 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:39.227996 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:34:39.228930 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:34:39.228930 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:39.228930 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:39.228930 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 464/928] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:34:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 10:34:39.233673 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a124d0d5-d1e1-4f7d-b67e-6fb80fcfb2a9 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:39.266290 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52466ee8-1802-40d0-873b-efb718cf5c5c None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:39.289296 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:39.314043 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d71ea05-9274-4a8a-9de2-d80e52816bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:39.315143 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d71ea05-9274-4a8a-9de2-d80e52816bb5 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:39.315759 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d71ea05-9274-4a8a-9de2-d80e52816bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:39.316194 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d71ea05-9274-4a8a-9de2-d80e52816bb5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:39.317016 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d71ea05-9274-4a8a-9de2-d80e52816bb5 None admin] RBAC: Adding query filter params (interface=public, service_id=b03c147cdbc94b38ba1e00ae0c3a10ce, region_id=RegionOne) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:39.317433 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d71ea05-9274-4a8a-9de2-d80e52816bb5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:39.319110 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d71ea05-9274-4a8a-9de2-d80e52816bb5 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:39.330532 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 465/929] 10.222.0.15 () {56 vars in 1251 bytes} [Sun Sep 11 10:34:39 2022] GET /identity/v3/endpoints?service_id=b03c147cdbc94b38ba1e00ae0c3a10ce&interface=public®ion_id=RegionOne => generated 194 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:39.332375 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d71ea05-9274-4a8a-9de2-d80e52816bb5 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:39.372733 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a124d0d5-d1e1-4f7d-b67e-6fb80fcfb2a9 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:39.415615 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172e2fc4-0ea7-4e09-99e4-8cacd3d7ddaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:39.416460 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172e2fc4-0ea7-4e09-99e4-8cacd3d7ddaa None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:39.416919 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172e2fc4-0ea7-4e09-99e4-8cacd3d7ddaa None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:39.417349 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172e2fc4-0ea7-4e09-99e4-8cacd3d7ddaa None admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:39.418232 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172e2fc4-0ea7-4e09-99e4-8cacd3d7ddaa None admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:39.418683 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172e2fc4-0ea7-4e09-99e4-8cacd3d7ddaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:39.420346 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172e2fc4-0ea7-4e09-99e4-8cacd3d7ddaa None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:39.439242 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 465/930] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:34:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:34:39.440462 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172e2fc4-0ea7-4e09-99e4-8cacd3d7ddaa None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:39.441537 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:39.465608 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d71ea05-9274-4a8a-9de2-d80e52816bb5 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:39.513982 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5504ef0b-e803-483d-8e45-d8020d975d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:39.514774 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5504ef0b-e803-483d-8e45-d8020d975d81 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:39.515150 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5504ef0b-e803-483d-8e45-d8020d975d81 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:39.515551 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5504ef0b-e803-483d-8e45-d8020d975d81 None admin] PATH_INFO: `/v3/services` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:39.516254 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5504ef0b-e803-483d-8e45-d8020d975d81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:39.518590 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5504ef0b-e803-483d-8e45-d8020d975d81 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:39.539991 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 466/931] 10.222.0.15 () {56 vars in 1087 bytes} [Sun Sep 11 10:34:39 2022] GET /identity/v3/services => generated 1497 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:34:39.556083 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:39.560377 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172e2fc4-0ea7-4e09-99e4-8cacd3d7ddaa None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:39.600727 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8553ed6e-468f-4c4a-935a-900eaf96d316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:39.601708 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8553ed6e-468f-4c4a-935a-900eaf96d316 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:39.602140 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8553ed6e-468f-4c4a-935a-900eaf96d316 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:39.602884 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8553ed6e-468f-4c4a-935a-900eaf96d316 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:39.611747 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8553ed6e-468f-4c4a-935a-900eaf96d316 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:39.613537 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8553ed6e-468f-4c4a-935a-900eaf96d316 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:39.628105 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-8553ed6e-468f-4c4a-935a-900eaf96d316 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 10:34:39.628105 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:39.628105 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:39.628105 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:39.628105 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:39.628105 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:39.628105 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:39.628105 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:39.628105 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:39.628105 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:39.628105 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:39.628105 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:39.628105 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 10:34:39.628105 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 10:34:39.628105 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 10:34:39.628105 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 10:34:39.630048 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:39.630048 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:39.630048 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 10:34:39.630048 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:34:39.630048 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 10:34:39.630048 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:34:39.630048 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:39.630048 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:39.630048 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:39.630048 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:39.630048 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:39.630048 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:39.630048 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:39.630048 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:39.630048 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:39.630048 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:39.630048 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:39.630048 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:39.631979 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:39.631979 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:39.631979 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 10:34:39.631979 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 10:34:39.631979 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 10:34:39.631979 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 10:34:39.631979 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 10:34:39.631979 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 10:34:39.631979 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 10:34:39.631979 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:39.639481 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 466/932] 10.222.0.15 () {56 vars in 1117 bytes} [Sun Sep 11 10:34:39 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:39.640467 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5504ef0b-e803-483d-8e45-d8020d975d81 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:39.764939 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086122b1-a0a9-4362-abc3-c161f814f6f7 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:39.765274 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086122b1-a0a9-4362-abc3-c161f814f6f7 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:39.765421 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086122b1-a0a9-4362-abc3-c161f814f6f7 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:39.766755 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 467/933] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:34:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 10:34:39.767889 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5504ef0b-e803-483d-8e45-d8020d975d81 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:39.774852 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ad218a-3177-41bd-a930-6f38ed94b6f9 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:39.775140 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ad218a-3177-41bd-a930-6f38ed94b6f9 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:39.775412 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ad218a-3177-41bd-a930-6f38ed94b6f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:39.796500 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-44ad218a-3177-41bd-a930-6f38ed94b6f9 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:34:39.797916 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-44ad218a-3177-41bd-a930-6f38ed94b6f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:34:39.815321 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb13402-7cac-43ce-adc3-553dcfa02808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:39.815986 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb13402-7cac-43ce-adc3-553dcfa02808 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:39.816271 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb13402-7cac-43ce-adc3-553dcfa02808 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:39.816548 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb13402-7cac-43ce-adc3-553dcfa02808 None admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:39.817544 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb13402-7cac-43ce-adc3-553dcfa02808 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:39.817855 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb13402-7cac-43ce-adc3-553dcfa02808 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:39.819843 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb13402-7cac-43ce-adc3-553dcfa02808 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:39.844572 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 467/934] 10.222.0.15 () {56 vars in 1149 bytes} [Sun Sep 11 10:34:39 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:34:39.854691 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb13402-7cac-43ce-adc3-553dcfa02808 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:39.862936 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:39.891290 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 468/935] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:34:39 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 10:34:39.969785 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb13402-7cac-43ce-adc3-553dcfa02808 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:40.006676 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2e34be-4a7a-4fb7-a413-bb094c55513a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:40.007928 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e34be-4a7a-4fb7-a413-bb094c55513a None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:40.008433 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e34be-4a7a-4fb7-a413-bb094c55513a None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:40.008909 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e34be-4a7a-4fb7-a413-bb094c55513a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:40.019843 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2e34be-4a7a-4fb7-a413-bb094c55513a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:40.022123 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2e34be-4a7a-4fb7-a413-bb094c55513a None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:40.037657 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-0e2e34be-4a7a-4fb7-a413-bb094c55513a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:40.037657 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:40.037657 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:40.037657 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:40.037657 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:40.037657 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:40.037657 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:40.037657 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:40.037657 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:40.037657 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:40.037657 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:40.037657 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:40.037657 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:34:40.037657 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:34:40.037657 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:34:40.037657 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:34:40.038628 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:40.038628 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:40.038628 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:40.038628 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:40.038628 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:40.038628 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:40.038628 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:40.038628 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:40.038628 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:40.038628 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:40.038628 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:40.038628 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:40.038628 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:40.038628 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:40.038628 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:40.038628 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:40.038628 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:34:40.039583 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:34:40.039583 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:40.039583 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:40.041773 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 468/936] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:34:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 10:34:40.047674 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ad218a-3177-41bd-a930-6f38ed94b6f9 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:40.145092 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:40.155299 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ad218a-3177-41bd-a930-6f38ed94b6f9 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:40.190143 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338c6767-f06e-4ef4-b537-482855c13d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:40.190799 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338c6767-f06e-4ef4-b537-482855c13d1d None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:40.191083 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338c6767-f06e-4ef4-b537-482855c13d1d None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:40.191360 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338c6767-f06e-4ef4-b537-482855c13d1d None admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:40.192085 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338c6767-f06e-4ef4-b537-482855c13d1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:40.192391 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338c6767-f06e-4ef4-b537-482855c13d1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:40.194451 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338c6767-f06e-4ef4-b537-482855c13d1d None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:40.204455 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2e34be-4a7a-4fb7-a413-bb094c55513a None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:40.206514 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 469/937] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:34:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:40.219119 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338c6767-f06e-4ef4-b537-482855c13d1d None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:40.271922 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:40.322515 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2e34be-4a7a-4fb7-a413-bb094c55513a None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:40.332173 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338c6767-f06e-4ef4-b537-482855c13d1d None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:40.359165 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0367dde9-f76b-495f-99c8-d3698f24f683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:40.360194 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0367dde9-f76b-495f-99c8-d3698f24f683 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:40.360650 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0367dde9-f76b-495f-99c8-d3698f24f683 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:40.361091 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0367dde9-f76b-495f-99c8-d3698f24f683 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:40.361837 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0367dde9-f76b-495f-99c8-d3698f24f683 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:40.363380 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0367dde9-f76b-495f-99c8-d3698f24f683 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:40.370519 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2338bb1-228f-45b5-b56f-e5cfeebedb5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:40.371234 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2338bb1-228f-45b5-b56f-e5cfeebedb5b None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:40.371634 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2338bb1-228f-45b5-b56f-e5cfeebedb5b None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:40.371982 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2338bb1-228f-45b5-b56f-e5cfeebedb5b None admin] PATH_INFO: `/v3/projects/service` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:40.374588 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-0367dde9-f76b-495f-99c8-d3698f24f683 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 11 10:34:40.374588 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:40.374588 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 10:34:40.374588 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 10:34:40.374588 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 10:34:40.374588 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 10:34:40.374588 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 10:34:40.374588 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 10:34:40.374588 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 11 10:34:40.374588 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:40.374588 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 10:34:40.374588 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:40.374588 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:40.374588 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:40.374588 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:40.374588 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:40.375537 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:40.375537 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:40.375537 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:40.375537 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:40.375537 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:40.375537 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:40.375537 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:40.375537 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 10:34:40.375537 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 10:34:40.375537 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 10:34:40.375537 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 10:34:40.375537 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:40.375537 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:40.375537 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:40.375537 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:40.375537 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:40.375537 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:40.376436 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:40.376436 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:40.376436 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:40.376436 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:40.376436 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:40.376436 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:40.376436 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:40.376436 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:40.376436 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:40.376436 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:40.376436 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 10:34:40.376436 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 10:34:40.376436 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 11 10:34:40.376436 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:40.384318 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2338bb1-228f-45b5-b56f-e5cfeebedb5b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:40.384318 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2338bb1-228f-45b5-b56f-e5cfeebedb5b None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:40.384963 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 469/938] 10.222.0.15 () {56 vars in 1132 bytes} [Sun Sep 11 10:34:40 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:40.385973 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0367dde9-f76b-495f-99c8-d3698f24f683 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:40.397778 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-e2338bb1-228f-45b5-b56f-e5cfeebedb5b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:34:40.397778 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:40.397778 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:40.397778 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:40.397778 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:40.397778 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:40.397778 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:40.397778 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:40.397778 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:40.397778 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:40.397778 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:40.397778 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:40.397778 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 10:34:40.397778 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 10:34:40.397778 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 10:34:40.398936 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 10:34:40.398936 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:40.398936 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:40.398936 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:40.398936 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:40.398936 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:40.398936 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:40.398936 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:40.398936 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:40.398936 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:40.398936 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:40.398936 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:40.398936 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:40.398936 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:40.398936 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:40.398936 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:40.398936 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:40.400079 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 10:34:40.400079 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 10:34:40.400079 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 10:34:40.400079 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 10:34:40.400079 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 10:34:40.400079 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 10:34:40.400079 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:34:40.400079 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:40.400946 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 470/939] 10.222.0.15 () {56 vars in 1111 bytes} [Sun Sep 11 10:34:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:40.401837 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a26225-ac85-4f37-be40-61dcf54a4cbf None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:40.402122 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a26225-ac85-4f37-be40-61dcf54a4cbf None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:40.402395 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a26225-ac85-4f37-be40-61dcf54a4cbf None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:40.405169 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 471/940] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:34:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:34:40.409384 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e124ec-130e-4957-b13b-f049b5b73bb7 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:40.409759 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e124ec-130e-4957-b13b-f049b5b73bb7 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:40.410129 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e124ec-130e-4957-b13b-f049b5b73bb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:40.431036 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-b4e124ec-130e-4957-b13b-f049b5b73bb7 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:34:40.432245 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-b4e124ec-130e-4957-b13b-f049b5b73bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:34:40.435074 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:40.499557 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0367dde9-f76b-495f-99c8-d3698f24f683 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:40.505238 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 472/941] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:34:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 10:34:40.507342 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e124ec-130e-4957-b13b-f049b5b73bb7 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:40.537651 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:40.554399 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acbc8dd-4d41-47a4-9021-3f9175ca2295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:40.555027 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acbc8dd-4d41-47a4-9021-3f9175ca2295 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:40.555309 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acbc8dd-4d41-47a4-9021-3f9175ca2295 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:40.555611 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acbc8dd-4d41-47a4-9021-3f9175ca2295 None admin] PATH_INFO: `/v3/services` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:40.556237 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acbc8dd-4d41-47a4-9021-3f9175ca2295 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:40.556540 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acbc8dd-4d41-47a4-9021-3f9175ca2295 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:40.557897 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acbc8dd-4d41-47a4-9021-3f9175ca2295 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:40.563873 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 470/942] 10.222.0.15 () {56 vars in 1126 bytes} [Sun Sep 11 10:34:40 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:40.569750 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acbc8dd-4d41-47a4-9021-3f9175ca2295 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:40.635747 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e124ec-130e-4957-b13b-f049b5b73bb7 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:40.672858 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11557d52-9158-4aa6-8d5a-aaff370aaa8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:40.673471 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11557d52-9158-4aa6-8d5a-aaff370aaa8a None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:40.673989 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11557d52-9158-4aa6-8d5a-aaff370aaa8a None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:40.674098 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11557d52-9158-4aa6-8d5a-aaff370aaa8a None admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:40.674842 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11557d52-9158-4aa6-8d5a-aaff370aaa8a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:40.675146 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11557d52-9158-4aa6-8d5a-aaff370aaa8a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:40.676663 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11557d52-9158-4aa6-8d5a-aaff370aaa8a None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:40.686320 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:40.690387 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 473/943] 10.222.0.15 () {56 vars in 1148 bytes} [Sun Sep 11 10:34:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:40.693022 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acbc8dd-4d41-47a4-9021-3f9175ca2295 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:40.696724 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11557d52-9158-4aa6-8d5a-aaff370aaa8a None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:40.741635 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d6abc4-8075-42c7-b8ab-818a3d2468f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:40.742255 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d6abc4-8075-42c7-b8ab-818a3d2468f9 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:40.742534 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d6abc4-8075-42c7-b8ab-818a3d2468f9 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:40.742808 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d6abc4-8075-42c7-b8ab-818a3d2468f9 None admin] PATH_INFO: `/v3/services` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:40.743450 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d6abc4-8075-42c7-b8ab-818a3d2468f9 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:40.743762 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d6abc4-8075-42c7-b8ab-818a3d2468f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:40.745245 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d6abc4-8075-42c7-b8ab-818a3d2468f9 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:40.751231 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 471/944] 10.222.0.15 () {56 vars in 1126 bytes} [Sun Sep 11 10:34:40 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:34:40.756409 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d6abc4-8075-42c7-b8ab-818a3d2468f9 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:40.817016 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11557d52-9158-4aa6-8d5a-aaff370aaa8a None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:40.821679 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:40.857806 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbea1c8-d8bc-47a1-97a4-1bd382d08ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:40.858472 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbea1c8-d8bc-47a1-97a4-1bd382d08ce4 None admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:40.858756 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbea1c8-d8bc-47a1-97a4-1bd382d08ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:40.859028 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbea1c8-d8bc-47a1-97a4-1bd382d08ce4 None admin] PATH_INFO: `/v3/projects/5e1f845f99f04202b83621cb65e0adc2/users/bb44e7aef0c7436d93a7233307b5f095/roles/55b167894e584c9b8303c60cf3a75a65` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:40.861216 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbea1c8-d8bc-47a1-97a4-1bd382d08ce4 None admin] RBAC: Authorizing `identity:create_grant(project_id=5e1f845f99f04202b83621cb65e0adc2, user_id=bb44e7aef0c7436d93a7233307b5f095, role_id=55b167894e584c9b8303c60cf3a75a65)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:40.862640 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbea1c8-d8bc-47a1-97a4-1bd382d08ce4 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:40.877127 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 474/945] 10.222.0.15 () {58 vars in 1439 bytes} [Sun Sep 11 10:34:40 2022] PUT /identity/v3/projects/5e1f845f99f04202b83621cb65e0adc2/users/bb44e7aef0c7436d93a7233307b5f095/roles/55b167894e584c9b8303c60cf3a75a65 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:34:40.880430 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbea1c8-d8bc-47a1-97a4-1bd382d08ce4 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:40.880572 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d6abc4-8075-42c7-b8ab-818a3d2468f9 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:40.922642 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580cb292-f7dc-4761-97f4-cee12e45a011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:40.923302 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580cb292-f7dc-4761-97f4-cee12e45a011 None admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:40.923665 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580cb292-f7dc-4761-97f4-cee12e45a011 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:40.923941 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580cb292-f7dc-4761-97f4-cee12e45a011 None admin] PATH_INFO: `/v3/endpoints` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:40.924677 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580cb292-f7dc-4761-97f4-cee12e45a011 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:40.926122 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580cb292-f7dc-4761-97f4-cee12e45a011 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:40.958724 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 472/946] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:34:40 2022] POST /identity/v3/endpoints => generated 347 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 10:34:40.986761 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:41.040163 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbea1c8-d8bc-47a1-97a4-1bd382d08ce4 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:41.091389 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7588841-bd72-4367-8b39-7cc063bb55e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:41.092099 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7588841-bd72-4367-8b39-7cc063bb55e0 None admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:41.092475 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7588841-bd72-4367-8b39-7cc063bb55e0 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:41.092843 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7588841-bd72-4367-8b39-7cc063bb55e0 None admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:41.093589 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7588841-bd72-4367-8b39-7cc063bb55e0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:41.095131 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7588841-bd72-4367-8b39-7cc063bb55e0 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:41.097930 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: WARNING keystone.server.flask.common [None req-c7588841-bd72-4367-8b39-7cc063bb55e0 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 11 10:34:41.120759 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.sql.core [None req-c7588841-bd72-4367-8b39-7cc063bb55e0 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 11 10:34:41.120759 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 11 10:34:41.120759 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [parameters: {'id': 'cfb665ebbae8403baaaa33d1a18e9cb4', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 11 10:34:41.120759 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=90654) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 11 10:34:41.138066 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-c7588841-bd72-4367-8b39-7cc063bb55e0 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id cfb665ebbae8403baaaa33d1a18e9cb4.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id cfb665ebbae8403baaaa33d1a18e9cb4. Sep 11 10:34:41.138066 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:41.138066 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 11 10:34:41.138066 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 11 10:34:41.138066 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 11 10:34:41.138066 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 11 10:34:41.138066 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 11 10:34:41.138066 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application result = self._query(query) Sep 11 10:34:41.138066 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 11 10:34:41.138066 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application conn.query(q) Sep 11 10:34:41.138066 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 11 10:34:41.138066 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 11 10:34:41.138066 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 11 10:34:41.139142 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application result.read() Sep 11 10:34:41.139142 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 11 10:34:41.139142 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 11 10:34:41.139142 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 11 10:34:41.139142 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 11 10:34:41.139142 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 11 10:34:41.139142 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 11 10:34:41.139142 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 11 10:34:41.139142 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 11 10:34:41.139142 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 11 10:34:41.139142 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:41.139142 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 11 10:34:41.139142 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:41.139142 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:41.139142 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 11 10:34:41.139142 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 11 10:34:41.139142 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 11 10:34:41.139142 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 11 10:34:41.140363 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 11 10:34:41.140363 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application next(self.gen) Sep 11 10:34:41.140363 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 11 10:34:41.140363 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application yield resource Sep 11 10:34:41.140363 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 11 10:34:41.140363 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application next(self.gen) Sep 11 10:34:41.140363 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 11 10:34:41.140363 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application self.session.rollback() Sep 11 10:34:41.140363 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 11 10:34:41.140363 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application self.force_reraise() Sep 11 10:34:41.140363 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 11 10:34:41.140363 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise self.value Sep 11 10:34:41.140363 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 11 10:34:41.140363 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 11 10:34:41.140363 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 11 10:34:41.140363 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application session.commit() Sep 11 10:34:41.140363 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 11 10:34:41.141480 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 11 10:34:41.141480 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 11 10:34:41.141480 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application self._prepare_impl() Sep 11 10:34:41.141480 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 11 10:34:41.141480 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application self.session.flush() Sep 11 10:34:41.141480 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 11 10:34:41.141480 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application self._flush(objects) Sep 11 10:34:41.141480 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 11 10:34:41.141480 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 11 10:34:41.141480 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 11 10:34:41.141480 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application compat.raise_( Sep 11 10:34:41.141480 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 11 10:34:41.141480 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception Sep 11 10:34:41.141480 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 11 10:34:41.141480 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application flush_context.execute() Sep 11 10:34:41.141480 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 11 10:34:41.141480 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rec.execute(self) Sep 11 10:34:41.142605 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 11 10:34:41.142605 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 11 10:34:41.142605 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 11 10:34:41.142605 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 11 10:34:41.142605 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 11 10:34:41.142605 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 11 10:34:41.142605 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 11 10:34:41.142605 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 11 10:34:41.142605 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 11 10:34:41.142605 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 11 10:34:41.142605 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 11 10:34:41.142605 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 11 10:34:41.142605 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 11 10:34:41.142605 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 11 10:34:41.142605 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 11 10:34:41.143706 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 11 10:34:41.143706 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 11 10:34:41.143706 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception Sep 11 10:34:41.143706 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 11 10:34:41.143706 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 11 10:34:41.143706 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 11 10:34:41.143706 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 11 10:34:41.143706 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 11 10:34:41.143706 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application result = self._query(query) Sep 11 10:34:41.143706 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 11 10:34:41.143706 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application conn.query(q) Sep 11 10:34:41.143706 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 11 10:34:41.143706 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 11 10:34:41.143706 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 11 10:34:41.143706 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application result.read() Sep 11 10:34:41.143706 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 11 10:34:41.143706 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 11 10:34:41.144808 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 11 10:34:41.144808 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 11 10:34:41.144808 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 11 10:34:41.144808 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 11 10:34:41.144808 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 11 10:34:41.144808 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 11 10:34:41.144808 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 11 10:34:41.144808 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 11 10:34:41.144808 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [parameters: {'id': 'cfb665ebbae8403baaaa33d1a18e9cb4', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 11 10:34:41.144808 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 11 10:34:41.144808 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:41.144808 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 10:34:41.144808 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:41.144808 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:41.144808 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 11 10:34:41.145775 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 11 10:34:41.145775 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 11 10:34:41.145775 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 11 10:34:41.145775 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 11 10:34:41.145775 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:41.145775 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 10:34:41.145775 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:41.145775 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:41.145775 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:41.145775 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:41.145775 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:41.145775 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:41.145775 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:41.145775 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:41.145775 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:41.145775 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:41.145775 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:41.146920 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:41.146920 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 11 10:34:41.146920 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 11 10:34:41.146920 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:41.146920 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:41.146920 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 11 10:34:41.146920 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 11 10:34:41.146920 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id cfb665ebbae8403baaaa33d1a18e9cb4. Sep 11 10:34:41.146920 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:41.152645 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 475/947] 10.222.0.15 () {60 vars in 1140 bytes} [Sun Sep 11 10:34:40 2022] POST /identity/v3/projects => generated 262 bytes in 270 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:41.153954 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580cb292-f7dc-4761-97f4-cee12e45a011 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:41.273039 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580cb292-f7dc-4761-97f4-cee12e45a011 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:41.280596 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:41.312459 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9ec66d-d21b-44d8-932f-f11ee43c1e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:41.313314 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9ec66d-d21b-44d8-932f-f11ee43c1e74 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:41.313586 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9ec66d-d21b-44d8-932f-f11ee43c1e74 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:41.313922 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9ec66d-d21b-44d8-932f-f11ee43c1e74 None admin] PATH_INFO: `/v3/projects/service` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:41.322004 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9ec66d-d21b-44d8-932f-f11ee43c1e74 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:41.323569 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9ec66d-d21b-44d8-932f-f11ee43c1e74 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:41.335984 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-be9ec66d-d21b-44d8-932f-f11ee43c1e74 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:34:41.335984 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:41.335984 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:41.335984 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:41.335984 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:41.335984 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:41.335984 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:41.335984 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:41.335984 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:41.335984 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:41.335984 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:41.335984 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:41.335984 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 10:34:41.335984 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 10:34:41.335984 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 10:34:41.337097 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 10:34:41.337097 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:41.337097 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:41.337097 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:41.337097 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:41.337097 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:41.337097 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:41.337097 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:41.337097 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:41.337097 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:41.337097 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:41.337097 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:41.337097 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:41.337097 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:41.337097 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:41.337097 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:41.337097 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:41.338218 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 10:34:41.338218 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 10:34:41.338218 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 10:34:41.338218 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 10:34:41.338218 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 10:34:41.338218 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 10:34:41.338218 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:34:41.338218 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:41.339601 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 473/948] 10.222.0.15 () {56 vars in 1111 bytes} [Sun Sep 11 10:34:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:41.345131 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7588841-bd72-4367-8b39-7cc063bb55e0 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:41.454312 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7588841-bd72-4367-8b39-7cc063bb55e0 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:41.468752 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:41.492599 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad9463b-3063-4675-b1d7-1e1db3d03ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:41.493500 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad9463b-3063-4675-b1d7-1e1db3d03ec8 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:41.493948 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad9463b-3063-4675-b1d7-1e1db3d03ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:41.494387 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad9463b-3063-4675-b1d7-1e1db3d03ec8 None admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:41.495218 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad9463b-3063-4675-b1d7-1e1db3d03ec8 None admin] RBAC: Adding query filter params (name=service) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:41.495687 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad9463b-3063-4675-b1d7-1e1db3d03ec8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:41.497454 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad9463b-3063-4675-b1d7-1e1db3d03ec8 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:41.518915 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 476/949] 10.222.0.15 () {56 vars in 1112 bytes} [Sun Sep 11 10:34:41 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:41.830262 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a90752-e7e6-4dae-8693-68906f7d8a2e None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:41.830784 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a90752-e7e6-4dae-8693-68906f7d8a2e None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:41.830784 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a90752-e7e6-4dae-8693-68906f7d8a2e None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:41.832309 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 474/950] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:34:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:34:41.838559 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664e1d38-bcf8-4499-968c-82b4c3501749 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:41.839023 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664e1d38-bcf8-4499-968c-82b4c3501749 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:41.839104 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664e1d38-bcf8-4499-968c-82b4c3501749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:41.861590 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-664e1d38-bcf8-4499-968c-82b4c3501749 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:34:41.863174 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-664e1d38-bcf8-4499-968c-82b4c3501749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:34:41.934376 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 477/951] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:34:41 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 10:34:42.200036 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a90752-e7e6-4dae-8693-68906f7d8a2e None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:42.307848 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a90752-e7e6-4dae-8693-68906f7d8a2e None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:42.344085 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c91297-6e25-48c6-841a-2a6e5f57b23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:42.344727 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c91297-6e25-48c6-841a-2a6e5f57b23a None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:42.345008 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c91297-6e25-48c6-841a-2a6e5f57b23a None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:42.345291 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c91297-6e25-48c6-841a-2a6e5f57b23a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:42.345875 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c91297-6e25-48c6-841a-2a6e5f57b23a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:42.347231 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c91297-6e25-48c6-841a-2a6e5f57b23a None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:42.356651 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-f4c91297-6e25-48c6-841a-2a6e5f57b23a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 10:34:42.356651 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:42.356651 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 10:34:42.356651 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 10:34:42.356651 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 10:34:42.356651 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 10:34:42.356651 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 10:34:42.356651 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 10:34:42.356651 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 10:34:42.356651 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:42.356651 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 10:34:42.356651 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:42.356651 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:42.356651 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:42.356651 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:42.356651 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:42.356651 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:42.357646 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:42.357646 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:42.357646 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:42.357646 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:42.357646 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:42.357646 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:42.357646 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 10:34:42.357646 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 10:34:42.357646 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 10:34:42.357646 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 10:34:42.357646 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:42.357646 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:42.357646 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:42.357646 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:42.357646 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:42.357646 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:42.358699 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:42.358699 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:42.358699 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:42.358699 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:42.358699 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:42.358699 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:42.358699 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:42.358699 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:42.358699 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:42.358699 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:42.358699 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 10:34:42.358699 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 10:34:42.358699 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 10:34:42.358699 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:42.358699 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 475/952] 10.222.0.15 () {56 vars in 1126 bytes} [Sun Sep 11 10:34:42 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:42.367207 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664e1d38-bcf8-4499-968c-82b4c3501749 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:42.472291 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664e1d38-bcf8-4499-968c-82b4c3501749 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:42.474115 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:42.511749 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f371c4ca-1a7a-4829-98a3-e8aaf99ea92d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:42.512807 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f371c4ca-1a7a-4829-98a3-e8aaf99ea92d None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:42.513302 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f371c4ca-1a7a-4829-98a3-e8aaf99ea92d None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:42.513788 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f371c4ca-1a7a-4829-98a3-e8aaf99ea92d None admin] PATH_INFO: `/v3/services` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:42.514628 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f371c4ca-1a7a-4829-98a3-e8aaf99ea92d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:42.515146 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f371c4ca-1a7a-4829-98a3-e8aaf99ea92d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:42.516831 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f371c4ca-1a7a-4829-98a3-e8aaf99ea92d None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:42.522934 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 478/953] 10.222.0.15 () {56 vars in 1122 bytes} [Sun Sep 11 10:34:42 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:42.528983 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c91297-6e25-48c6-841a-2a6e5f57b23a None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:42.560763 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:42.634925 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c91297-6e25-48c6-841a-2a6e5f57b23a None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:42.670608 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b405c02-9cca-4b27-bea6-3d178f3a4bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:42.671275 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b405c02-9cca-4b27-bea6-3d178f3a4bdf None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:42.671620 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b405c02-9cca-4b27-bea6-3d178f3a4bdf None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:42.671896 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b405c02-9cca-4b27-bea6-3d178f3a4bdf None admin] PATH_INFO: `/v3/services` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:42.672567 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b405c02-9cca-4b27-bea6-3d178f3a4bdf None admin] RBAC: Adding query filter params (type=object-store) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:42.672871 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b405c02-9cca-4b27-bea6-3d178f3a4bdf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:42.674233 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b405c02-9cca-4b27-bea6-3d178f3a4bdf None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:42.679917 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 476/954] 10.222.0.15 () {56 vars in 1122 bytes} [Sun Sep 11 10:34:42 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:42.685230 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f371c4ca-1a7a-4829-98a3-e8aaf99ea92d None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:42.800889 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f371c4ca-1a7a-4829-98a3-e8aaf99ea92d None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:42.837752 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c1ca5a-bae8-4777-911f-ade792ac6743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:42.838363 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c1ca5a-bae8-4777-911f-ade792ac6743 None admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:42.838643 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c1ca5a-bae8-4777-911f-ade792ac6743 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:42.839248 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c1ca5a-bae8-4777-911f-ade792ac6743 None admin] PATH_INFO: `/v3/endpoints` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:42.839608 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c1ca5a-bae8-4777-911f-ade792ac6743 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:42.841170 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c1ca5a-bae8-4777-911f-ade792ac6743 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:42.865606 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 479/955] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:34:42 2022] POST /identity/v3/endpoints => generated 349 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:34:43.210057 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeb3a7d-e945-40ae-bd8f-bc8a51195bae None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:43.210057 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeb3a7d-e945-40ae-bd8f-bc8a51195bae None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:43.210057 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeb3a7d-e945-40ae-bd8f-bc8a51195bae None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:43.210948 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 477/956] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:34:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:34:43.219760 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76436d53-bf86-48d3-99e4-42d3674f242e None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:43.219760 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76436d53-bf86-48d3-99e4-42d3674f242e None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:43.219760 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76436d53-bf86-48d3-99e4-42d3674f242e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:43.237191 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-76436d53-bf86-48d3-99e4-42d3674f242e None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:34:43.238756 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-76436d53-bf86-48d3-99e4-42d3674f242e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:34:43.263887 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:43.329867 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 480/957] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:34:43 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 10:34:43.366031 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c91d62b-5474-4c4b-8fb0-a1a99ce8b52c None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:43.366311 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c91d62b-5474-4c4b-8fb0-a1a99ce8b52c None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:43.366590 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c91d62b-5474-4c4b-8fb0-a1a99ce8b52c None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:43.367957 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 478/958] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:34:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:34:43.373499 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f3128-be79-40cc-86c5-92221fa87051 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:43.373786 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f3128-be79-40cc-86c5-92221fa87051 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:43.374060 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f3128-be79-40cc-86c5-92221fa87051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:43.393329 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-b63f3128-be79-40cc-86c5-92221fa87051 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:34:43.394510 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-b63f3128-be79-40cc-86c5-92221fa87051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:34:43.465882 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 481/959] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:34:43 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 10:34:43.612818 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c91d62b-5474-4c4b-8fb0-a1a99ce8b52c None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:43.714695 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:43.724398 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c91d62b-5474-4c4b-8fb0-a1a99ce8b52c None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:43.750084 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63f3128-be79-40cc-86c5-92221fa87051 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:43.765230 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3331f8-e04b-4795-a685-2642199e484a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:43.766003 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3331f8-e04b-4795-a685-2642199e484a None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:43.766284 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3331f8-e04b-4795-a685-2642199e484a None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:43.766556 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3331f8-e04b-4795-a685-2642199e484a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:43.780985 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3331f8-e04b-4795-a685-2642199e484a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:34:43.786485 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-2b3331f8-e04b-4795-a685-2642199e484a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 10:34:43.786485 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:43.786485 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:43.786485 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:43.786485 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:43.786485 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:43.786485 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:43.786485 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:43.786485 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:43.786485 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:43.786485 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:43.786485 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:43.786485 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:34:43.786485 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:34:43.786485 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:34:43.786485 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:34:43.787842 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:34:43.787842 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:34:43.787842 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:34:43.787842 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:34:43.787842 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:43.787842 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:43.787842 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:43.787842 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:43.787842 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:43.787842 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:43.787842 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:43.787842 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:43.787842 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:43.787842 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:43.787842 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:43.787842 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:43.788738 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:43.788738 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:43.788738 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:43.788738 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:43.788738 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:34:43.788738 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:34:43.788738 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:34:43.788738 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:34:43.788738 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:34:43.788738 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:34:43.788738 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 10:34:43.788738 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:43.791597 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 479/960] 10.222.0.15 () {56 vars in 1120 bytes} [Sun Sep 11 10:34:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:43.797996 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3331f8-e04b-4795-a685-2642199e484a None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:43.861037 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:43.891840 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63f3128-be79-40cc-86c5-92221fa87051 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:43.918159 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3331f8-e04b-4795-a685-2642199e484a None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:43.929955 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bc41a8-8b3f-423b-aa65-68d1b8af0cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:43.930663 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bc41a8-8b3f-423b-aa65-68d1b8af0cc4 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:43.930944 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bc41a8-8b3f-423b-aa65-68d1b8af0cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:43.931209 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bc41a8-8b3f-423b-aa65-68d1b8af0cc4 None admin] PATH_INFO: `/v3/services/compute` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:43.931829 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bc41a8-8b3f-423b-aa65-68d1b8af0cc4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:43.933191 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bc41a8-8b3f-423b-aa65-68d1b8af0cc4 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:43.941817 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-d1bc41a8-8b3f-423b-aa65-68d1b8af0cc4 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 11 10:34:43.941817 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:43.941817 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 10:34:43.941817 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 10:34:43.941817 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 10:34:43.941817 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 10:34:43.941817 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 10:34:43.941817 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 10:34:43.941817 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 11 10:34:43.941817 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:43.941817 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 10:34:43.941817 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:43.941817 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:43.941817 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:43.941817 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:43.941817 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:43.941817 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:43.942851 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:43.942851 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:43.942851 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:43.942851 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:43.942851 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:43.942851 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:43.942851 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 10:34:43.942851 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 10:34:43.942851 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 10:34:43.942851 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 10:34:43.942851 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:43.942851 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:43.942851 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:43.942851 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:43.942851 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:43.942851 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:43.944350 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:43.944350 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:43.944350 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:43.944350 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:43.944350 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:43.944350 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:43.944350 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:43.944350 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:43.944350 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:43.944350 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:43.944350 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 10:34:43.944350 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 10:34:43.944350 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 11 10:34:43.944350 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:43.948163 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 482/961] 10.222.0.15 () {56 vars in 1111 bytes} [Sun Sep 11 10:34:43 2022] GET /identity/v3/services/compute => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 10:34:43.949272 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f51b1-c90b-4f5f-9639-311019227660 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:43.949554 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f51b1-c90b-4f5f-9639-311019227660 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:43.949820 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f51b1-c90b-4f5f-9639-311019227660 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:43.951176 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 483/962] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:34:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:34:43.951745 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990f51b1-c90b-4f5f-9639-311019227660 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:43.962417 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f131bbfb-547a-4427-acbc-4ee52672c1c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:43.963502 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f131bbfb-547a-4427-acbc-4ee52672c1c1 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:43.963792 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f131bbfb-547a-4427-acbc-4ee52672c1c1 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:43.964076 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f131bbfb-547a-4427-acbc-4ee52672c1c1 None admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:43.965109 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f131bbfb-547a-4427-acbc-4ee52672c1c1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:43.965420 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f131bbfb-547a-4427-acbc-4ee52672c1c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:43.967125 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f131bbfb-547a-4427-acbc-4ee52672c1c1 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:43.980967 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 480/963] 10.222.0.15 () {56 vars in 1115 bytes} [Sun Sep 11 10:34:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:43.980967 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1affc9ee-4ce9-4ba8-9883-85a07224e72a None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:43.980967 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1affc9ee-4ce9-4ba8-9883-85a07224e72a None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:43.980967 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1affc9ee-4ce9-4ba8-9883-85a07224e72a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:43.996854 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-1affc9ee-4ce9-4ba8-9883-85a07224e72a None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:34:43.998041 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-1affc9ee-4ce9-4ba8-9883-85a07224e72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:34:44.010222 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:44.072276 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 481/964] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:34:43 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 10:34:44.073354 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1affc9ee-4ce9-4ba8-9883-85a07224e72a None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:44.086819 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990f51b1-c90b-4f5f-9639-311019227660 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:44.136323 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34187c8b-0378-4863-8278-3044f35a9c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:44.137453 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34187c8b-0378-4863-8278-3044f35a9c90 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:44.137800 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34187c8b-0378-4863-8278-3044f35a9c90 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:44.138139 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34187c8b-0378-4863-8278-3044f35a9c90 None admin] PATH_INFO: `/v3/services` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:44.139095 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34187c8b-0378-4863-8278-3044f35a9c90 None admin] RBAC: Adding query filter params (name=compute) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:44.139499 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34187c8b-0378-4863-8278-3044f35a9c90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:44.141617 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34187c8b-0378-4863-8278-3044f35a9c90 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:44.151575 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 484/965] 10.222.0.15 () {56 vars in 1112 bytes} [Sun Sep 11 10:34:43 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:44.155057 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34187c8b-0378-4863-8278-3044f35a9c90 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:44.168196 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:44.198475 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1affc9ee-4ce9-4ba8-9883-85a07224e72a None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:44.244070 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541b4e77-9ccf-405c-957a-bd06a345bedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:44.245788 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b4e77-9ccf-405c-957a-bd06a345bedf None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:44.246241 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b4e77-9ccf-405c-957a-bd06a345bedf None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:44.246659 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b4e77-9ccf-405c-957a-bd06a345bedf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:44.260441 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541b4e77-9ccf-405c-957a-bd06a345bedf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:44.262472 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541b4e77-9ccf-405c-957a-bd06a345bedf None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:44.271602 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:44.271833 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34187c8b-0378-4863-8278-3044f35a9c90 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:44.287618 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-541b4e77-9ccf-405c-957a-bd06a345bedf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:44.287618 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:44.287618 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:44.287618 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:44.287618 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:44.287618 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:44.287618 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:44.287618 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:44.287618 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:44.287618 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:44.287618 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:44.287618 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:44.287618 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:34:44.287618 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:34:44.287618 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:34:44.287618 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:34:44.288860 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:44.288860 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:44.288860 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:44.288860 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:44.288860 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:44.288860 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:44.288860 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:44.288860 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:44.288860 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:44.288860 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:44.288860 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:44.288860 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:44.288860 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:44.288860 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:44.288860 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:44.288860 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:44.288860 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:34:44.289955 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:34:44.289955 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:44.289955 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:44.295744 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 482/966] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:34:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:44.296668 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541b4e77-9ccf-405c-957a-bd06a345bedf None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:44.323741 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc36a7e-a275-4454-935a-e83de88b9475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:44.324623 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc36a7e-a275-4454-935a-e83de88b9475 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:44.325079 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc36a7e-a275-4454-935a-e83de88b9475 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:44.325540 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc36a7e-a275-4454-935a-e83de88b9475 None admin] PATH_INFO: `/v3/services` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:44.326347 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc36a7e-a275-4454-935a-e83de88b9475 None admin] RBAC: Adding query filter params (type=compute) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:44.326829 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc36a7e-a275-4454-935a-e83de88b9475 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:44.328398 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc36a7e-a275-4454-935a-e83de88b9475 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:44.339717 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 485/967] 10.222.0.15 () {56 vars in 1112 bytes} [Sun Sep 11 10:34:44 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:44.361414 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc36a7e-a275-4454-935a-e83de88b9475 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:44.416402 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541b4e77-9ccf-405c-957a-bd06a345bedf None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:44.456108 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:44.459106 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e58da34-d071-47d6-b076-cd5893d2028b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:44.460208 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e58da34-d071-47d6-b076-cd5893d2028b None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:44.460639 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e58da34-d071-47d6-b076-cd5893d2028b None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:44.461090 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e58da34-d071-47d6-b076-cd5893d2028b None admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:44.461898 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e58da34-d071-47d6-b076-cd5893d2028b None admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:44.462348 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e58da34-d071-47d6-b076-cd5893d2028b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:44.463929 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e58da34-d071-47d6-b076-cd5893d2028b None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:44.483724 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 483/968] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:34:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:44.484755 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e58da34-d071-47d6-b076-cd5893d2028b None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:44.499174 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc36a7e-a275-4454-935a-e83de88b9475 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:44.554896 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ebce81-37d3-4a02-a944-f84624667b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:44.556354 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ebce81-37d3-4a02-a944-f84624667b0d None admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:44.556823 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ebce81-37d3-4a02-a944-f84624667b0d None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:44.557349 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ebce81-37d3-4a02-a944-f84624667b0d None admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:44.557955 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ebce81-37d3-4a02-a944-f84624667b0d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:44.559400 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ebce81-37d3-4a02-a944-f84624667b0d None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:44.561860 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:44.575487 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.sql.core [None req-72ebce81-37d3-4a02-a944-f84624667b0d None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 11 10:34:44.575487 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 11 10:34:44.575487 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [parameters: {'user_id': 'b62255106c4d4648b2c3c463bdb28fb2', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 11 10:34:44.575487 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=90654) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 11 10:34:44.584173 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-72ebce81-37d3-4a02-a944-f84624667b0d None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 11 10:34:44.584173 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:44.584173 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 11 10:34:44.584173 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 11 10:34:44.584173 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 11 10:34:44.584173 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 11 10:34:44.584173 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 11 10:34:44.584173 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application result = self._query(query) Sep 11 10:34:44.584173 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 11 10:34:44.584173 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application conn.query(q) Sep 11 10:34:44.584173 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 11 10:34:44.584173 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 11 10:34:44.584173 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 11 10:34:44.584173 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application result.read() Sep 11 10:34:44.584173 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 11 10:34:44.585097 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 11 10:34:44.585097 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 11 10:34:44.585097 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 11 10:34:44.585097 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 11 10:34:44.585097 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 11 10:34:44.585097 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 11 10:34:44.585097 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 11 10:34:44.585097 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 11 10:34:44.585097 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:44.585097 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 11 10:34:44.585097 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:44.585097 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:44.585097 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 11 10:34:44.585097 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 11 10:34:44.585097 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 11 10:34:44.585097 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 11 10:34:44.585097 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 11 10:34:44.585097 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application next(self.gen) Sep 11 10:34:44.586088 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 11 10:34:44.586088 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application yield resource Sep 11 10:34:44.586088 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 11 10:34:44.586088 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application next(self.gen) Sep 11 10:34:44.586088 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 11 10:34:44.586088 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application self.session.rollback() Sep 11 10:34:44.586088 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 11 10:34:44.586088 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application self.force_reraise() Sep 11 10:34:44.586088 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 11 10:34:44.586088 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise self.value Sep 11 10:34:44.586088 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 11 10:34:44.586088 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 11 10:34:44.586088 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 11 10:34:44.586088 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application session.commit() Sep 11 10:34:44.586088 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 11 10:34:44.586088 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 11 10:34:44.586088 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 11 10:34:44.587088 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application self._prepare_impl() Sep 11 10:34:44.587088 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 11 10:34:44.587088 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application self.session.flush() Sep 11 10:34:44.587088 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 11 10:34:44.587088 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application self._flush(objects) Sep 11 10:34:44.587088 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 11 10:34:44.587088 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 11 10:34:44.587088 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 11 10:34:44.587088 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application compat.raise_( Sep 11 10:34:44.587088 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 11 10:34:44.587088 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception Sep 11 10:34:44.587088 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 11 10:34:44.587088 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application flush_context.execute() Sep 11 10:34:44.587088 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 11 10:34:44.587088 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rec.execute(self) Sep 11 10:34:44.587088 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 11 10:34:44.587088 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 11 10:34:44.588737 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 11 10:34:44.588737 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 11 10:34:44.588737 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 11 10:34:44.588737 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 11 10:34:44.588737 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 11 10:34:44.588737 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 11 10:34:44.588737 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 11 10:34:44.588737 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 11 10:34:44.588737 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 11 10:34:44.588737 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 11 10:34:44.588737 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 11 10:34:44.588737 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 11 10:34:44.588737 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 11 10:34:44.588737 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 11 10:34:44.588737 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 11 10:34:44.588737 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception Sep 11 10:34:44.589574 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 11 10:34:44.589574 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 11 10:34:44.589574 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 11 10:34:44.589574 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 11 10:34:44.589574 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 11 10:34:44.589574 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application result = self._query(query) Sep 11 10:34:44.589574 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 11 10:34:44.589574 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application conn.query(q) Sep 11 10:34:44.589574 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 11 10:34:44.589574 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 11 10:34:44.589574 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 11 10:34:44.589574 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application result.read() Sep 11 10:34:44.589574 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 11 10:34:44.589574 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 11 10:34:44.589574 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 11 10:34:44.589574 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 11 10:34:44.591279 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 11 10:34:44.591279 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 11 10:34:44.591279 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 11 10:34:44.591279 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 11 10:34:44.591279 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 11 10:34:44.591279 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 11 10:34:44.591279 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [parameters: {'user_id': 'b62255106c4d4648b2c3c463bdb28fb2', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 11 10:34:44.591279 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 11 10:34:44.591279 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:44.591279 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 10:34:44.591279 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:44.591279 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:44.591279 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:44.591279 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:44.591279 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:44.593201 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:44.593201 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:44.593201 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:44.593201 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:44.593201 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:44.593201 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:44.593201 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:44.593201 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 11 10:34:44.593201 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 11 10:34:44.593201 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:44.593201 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:44.593201 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 10:34:44.593201 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:34:44.593201 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 10:34:44.593201 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:34:44.593201 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 11 10:34:44.593201 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 11 10:34:44.595411 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 11 10:34:44.595411 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 11 10:34:44.595411 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 11 10:34:44.595411 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 11 10:34:44.595411 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 11 10:34:44.595411 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:44.597851 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 486/969] 10.222.0.15 () {60 vars in 1131 bytes} [Sun Sep 11 10:34:44 2022] POST /identity/v3/users => generated 160 bytes in 237 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 11 10:34:44.608061 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ebce81-37d3-4a02-a944-f84624667b0d None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:44.620870 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e58da34-d071-47d6-b076-cd5893d2028b None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:44.666414 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2da3c2-78f3-425d-962e-2425aef7d6d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:44.667702 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2da3c2-78f3-425d-962e-2425aef7d6d5 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:44.668278 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2da3c2-78f3-425d-962e-2425aef7d6d5 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:44.668745 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2da3c2-78f3-425d-962e-2425aef7d6d5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:44.677972 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2da3c2-78f3-425d-962e-2425aef7d6d5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:44.680025 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2da3c2-78f3-425d-962e-2425aef7d6d5 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:44.692968 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-fd2da3c2-78f3-425d-962e-2425aef7d6d5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 10:34:44.692968 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:44.692968 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:44.692968 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:44.692968 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:44.692968 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:44.692968 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:44.692968 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:44.692968 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:44.692968 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:44.692968 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:44.692968 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:44.692968 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 10:34:44.692968 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 10:34:44.692968 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 10:34:44.692968 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 10:34:44.694448 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:44.694448 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:44.694448 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 10:34:44.694448 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:34:44.694448 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 10:34:44.694448 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:34:44.694448 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:44.694448 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:44.694448 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:44.694448 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:44.694448 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:44.694448 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:44.694448 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:44.694448 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:44.694448 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:44.694448 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:44.694448 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:44.694448 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:44.695386 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:44.695386 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:44.695386 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 10:34:44.695386 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 10:34:44.695386 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 10:34:44.695386 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 10:34:44.695386 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 10:34:44.695386 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 10:34:44.695386 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 10:34:44.695386 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:44.698231 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 484/970] 10.222.0.15 () {56 vars in 1117 bytes} [Sun Sep 11 10:34:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:44.707528 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2da3c2-78f3-425d-962e-2425aef7d6d5 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:44.742240 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ebce81-37d3-4a02-a944-f84624667b0d None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:44.782462 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:44.791142 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5988514e-8874-414e-8505-54c9fcc9a383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:44.792156 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5988514e-8874-414e-8505-54c9fcc9a383 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:44.792746 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5988514e-8874-414e-8505-54c9fcc9a383 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:44.793213 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5988514e-8874-414e-8505-54c9fcc9a383 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:44.802948 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5988514e-8874-414e-8505-54c9fcc9a383 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:44.805585 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5988514e-8874-414e-8505-54c9fcc9a383 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:44.821987 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-5988514e-8874-414e-8505-54c9fcc9a383 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 11 10:34:44.821987 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:44.821987 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:44.821987 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:44.821987 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:44.821987 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:44.821987 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:44.821987 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:44.821987 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:44.821987 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:44.821987 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:44.821987 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:44.821987 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 10:34:44.821987 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 10:34:44.821987 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 10:34:44.821987 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 10:34:44.823026 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:44.823026 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:44.823026 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 10:34:44.823026 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:34:44.823026 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 10:34:44.823026 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:34:44.823026 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:44.823026 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:44.823026 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:44.823026 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:44.823026 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:44.823026 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:44.823026 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:44.823026 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:44.823026 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:44.823026 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:44.823026 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:44.823026 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:44.829531 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:44.829531 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:44.829531 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 10:34:44.829531 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 10:34:44.829531 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 10:34:44.829531 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 10:34:44.829531 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 10:34:44.829531 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 10:34:44.829531 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 11 10:34:44.829531 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:44.829531 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 487/971] 10.222.0.15 () {56 vars in 1099 bytes} [Sun Sep 11 10:34:44 2022] GET /identity/v3/users/cinder => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:44.831954 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5988514e-8874-414e-8505-54c9fcc9a383 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:44.836322 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2da3c2-78f3-425d-962e-2425aef7d6d5 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:44.863932 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:44.884011 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9531f9-c851-4b1f-8592-306e23ea8a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:44.884997 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9531f9-c851-4b1f-8592-306e23ea8a6e None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:44.885536 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9531f9-c851-4b1f-8592-306e23ea8a6e None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:44.886052 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9531f9-c851-4b1f-8592-306e23ea8a6e None admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:44.886977 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9531f9-c851-4b1f-8592-306e23ea8a6e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:44.887650 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9531f9-c851-4b1f-8592-306e23ea8a6e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:44.890001 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9531f9-c851-4b1f-8592-306e23ea8a6e None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:44.915790 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 485/972] 10.222.0.15 () {56 vars in 1149 bytes} [Sun Sep 11 10:34:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:34:44.923536 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9531f9-c851-4b1f-8592-306e23ea8a6e None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:44.949953 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5988514e-8874-414e-8505-54c9fcc9a383 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:44.993893 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5955bcf1-f3bd-4d6c-abc9-856a34bb7cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:44.994638 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5955bcf1-f3bd-4d6c-abc9-856a34bb7cd1 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:44.994991 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5955bcf1-f3bd-4d6c-abc9-856a34bb7cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:44.995446 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5955bcf1-f3bd-4d6c-abc9-856a34bb7cd1 None admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:44.996412 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5955bcf1-f3bd-4d6c-abc9-856a34bb7cd1 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:44.996796 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5955bcf1-f3bd-4d6c-abc9-856a34bb7cd1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:44.998350 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5955bcf1-f3bd-4d6c-abc9-856a34bb7cd1 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:45.037968 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9531f9-c851-4b1f-8592-306e23ea8a6e None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:45.046822 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 488/973] 10.222.0.15 () {56 vars in 1101 bytes} [Sun Sep 11 10:34:44 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:45.076855 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f511473e-7c20-4b81-baf9-4a48a6fb6f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:45.077483 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f511473e-7c20-4b81-baf9-4a48a6fb6f3b None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:45.077762 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f511473e-7c20-4b81-baf9-4a48a6fb6f3b None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:45.078034 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f511473e-7c20-4b81-baf9-4a48a6fb6f3b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:45.095445 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f9a6d5-18f1-4c65-9970-9e9109f0267d None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:45.095752 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f9a6d5-18f1-4c65-9970-9e9109f0267d None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:45.096029 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f9a6d5-18f1-4c65-9970-9e9109f0267d None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:45.099081 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 489/974] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:34:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 10:34:45.105596 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380aee5b-21a3-4eee-a794-9e5b85f9b674 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:45.105934 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380aee5b-21a3-4eee-a794-9e5b85f9b674 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:45.106275 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380aee5b-21a3-4eee-a794-9e5b85f9b674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:45.132010 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:45.134536 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-380aee5b-21a3-4eee-a794-9e5b85f9b674 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:34:45.136327 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-380aee5b-21a3-4eee-a794-9e5b85f9b674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:34:45.164279 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f511473e-7c20-4b81-baf9-4a48a6fb6f3b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:45.165870 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f511473e-7c20-4b81-baf9-4a48a6fb6f3b None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:45.192724 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:45.216518 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 490/975] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:34:45 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 10:34:45.297155 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 486/976] 10.222.0.15 () {58 vars in 1282 bytes} [Sun Sep 11 10:34:44 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 10:34:45.302990 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380aee5b-21a3-4eee-a794-9e5b85f9b674 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:45.415692 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380aee5b-21a3-4eee-a794-9e5b85f9b674 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:45.450595 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbc82d5-129b-4111-aa67-584818dbb737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:45.451223 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbc82d5-129b-4111-aa67-584818dbb737 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:45.451536 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbc82d5-129b-4111-aa67-584818dbb737 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:45.451810 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbc82d5-129b-4111-aa67-584818dbb737 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:45.462239 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbc82d5-129b-4111-aa67-584818dbb737 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:45.463755 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbc82d5-129b-4111-aa67-584818dbb737 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:45.478772 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-bfbc82d5-129b-4111-aa67-584818dbb737 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:45.478772 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:45.478772 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:45.478772 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:45.478772 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:45.478772 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:45.478772 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:45.478772 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:45.478772 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:45.478772 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:45.478772 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:45.478772 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:45.478772 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:34:45.478772 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:34:45.478772 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:34:45.478772 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:34:45.479777 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:45.479777 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:45.479777 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:45.479777 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:45.479777 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:45.479777 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:45.479777 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:45.479777 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:45.479777 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:45.479777 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:45.479777 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:45.479777 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:45.479777 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:45.479777 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:45.479777 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:45.479777 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:45.479777 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:34:45.480704 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:34:45.480704 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:45.480704 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:45.480704 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 491/977] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:34:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:45.485666 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f511473e-7c20-4b81-baf9-4a48a6fb6f3b None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:45.497627 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbc82d5-129b-4111-aa67-584818dbb737 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:45.530646 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:45.567716 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:45.594223 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f511473e-7c20-4b81-baf9-4a48a6fb6f3b None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:45.606245 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbc82d5-129b-4111-aa67-584818dbb737 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:45.628811 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52ecee3-b26b-4410-9b65-d3afa6a1d4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:45.629412 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52ecee3-b26b-4410-9b65-d3afa6a1d4de None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:45.629688 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52ecee3-b26b-4410-9b65-d3afa6a1d4de None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:45.629954 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52ecee3-b26b-4410-9b65-d3afa6a1d4de None admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:45.630670 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52ecee3-b26b-4410-9b65-d3afa6a1d4de None admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:45.631759 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52ecee3-b26b-4410-9b65-d3afa6a1d4de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:45.633263 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52ecee3-b26b-4410-9b65-d3afa6a1d4de None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:45.642769 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f5f3d2-abce-4262-92b3-a5e1c7625b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:45.643404 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5f3d2-abce-4262-92b3-a5e1c7625b51 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:45.643811 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5f3d2-abce-4262-92b3-a5e1c7625b51 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:45.644088 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5f3d2-abce-4262-92b3-a5e1c7625b51 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:45.644668 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f5f3d2-abce-4262-92b3-a5e1c7625b51 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:45.650860 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 487/978] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:34:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:34:45.651007 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f5f3d2-abce-4262-92b3-a5e1c7625b51 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:45.651764 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52ecee3-b26b-4410-9b65-d3afa6a1d4de None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:45.660027 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-a1f5f3d2-abce-4262-92b3-a5e1c7625b51 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 10:34:45.660027 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:45.660027 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 10:34:45.660027 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 10:34:45.660027 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 10:34:45.660027 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 10:34:45.660027 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 10:34:45.660027 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 10:34:45.660027 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 10:34:45.660027 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:45.660027 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 10:34:45.660027 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:45.660027 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:45.660027 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:45.660027 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:45.660027 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:45.660027 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:45.661078 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:45.661078 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:45.661078 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:45.661078 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:45.661078 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:45.661078 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:45.661078 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 10:34:45.661078 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 10:34:45.661078 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 10:34:45.661078 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 10:34:45.661078 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:45.661078 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:45.661078 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:45.661078 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:45.661078 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:45.661078 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:45.662022 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:45.662022 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:45.662022 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:45.662022 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:45.662022 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:45.662022 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:45.662022 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:45.662022 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:45.662022 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:45.662022 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:45.662022 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 10:34:45.662022 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 10:34:45.662022 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 10:34:45.662022 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:45.663952 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 492/979] 10.222.0.15 () {56 vars in 1126 bytes} [Sun Sep 11 10:34:45 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 10:34:45.667290 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f5f3d2-abce-4262-92b3-a5e1c7625b51 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:45.764547 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52ecee3-b26b-4410-9b65-d3afa6a1d4de None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:45.766378 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:45.773190 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f5f3d2-abce-4262-92b3-a5e1c7625b51 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:45.804671 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8a2201-23ea-4c01-bc74-bd68eab4caef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:45.805677 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8a2201-23ea-4c01-bc74-bd68eab4caef None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:45.806100 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8a2201-23ea-4c01-bc74-bd68eab4caef None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:45.806500 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8a2201-23ea-4c01-bc74-bd68eab4caef None admin] PATH_INFO: `/v3/projects/service` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:45.807337 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c97e00-dada-43bc-8666-e362b6a29c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:45.807980 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c97e00-dada-43bc-8666-e362b6a29c8e None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:45.808259 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c97e00-dada-43bc-8666-e362b6a29c8e None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:45.808531 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c97e00-dada-43bc-8666-e362b6a29c8e None admin] PATH_INFO: `/v3/services` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:45.809148 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c97e00-dada-43bc-8666-e362b6a29c8e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:45.809449 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c97e00-dada-43bc-8666-e362b6a29c8e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:45.810412 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:45.810806 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c97e00-dada-43bc-8666-e362b6a29c8e None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:45.816310 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8a2201-23ea-4c01-bc74-bd68eab4caef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:45.817989 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8a2201-23ea-4c01-bc74-bd68eab4caef None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:45.820943 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 493/980] 10.222.0.15 () {56 vars in 1122 bytes} [Sun Sep 11 10:34:45 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:34:45.821655 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c97e00-dada-43bc-8666-e362b6a29c8e None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:45.831774 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-0f8a2201-23ea-4c01-bc74-bd68eab4caef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:34:45.831774 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:45.831774 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:45.831774 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:45.831774 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:45.831774 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:45.831774 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:45.831774 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:45.831774 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:45.831774 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:45.831774 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:45.831774 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:45.831774 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 10:34:45.831774 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 10:34:45.831774 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 10:34:45.832661 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 10:34:45.832661 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:45.832661 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:45.832661 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:45.832661 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:45.832661 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:45.832661 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:45.832661 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:45.832661 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:45.832661 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:45.832661 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:45.832661 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:45.832661 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:45.832661 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:45.832661 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:45.832661 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:45.832661 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:45.833699 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 10:34:45.833699 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 10:34:45.833699 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 10:34:45.833699 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 10:34:45.833699 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 10:34:45.833699 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 10:34:45.833699 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:34:45.833699 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:45.835775 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 488/981] 10.222.0.15 () {56 vars in 1111 bytes} [Sun Sep 11 10:34:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:45.841303 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8a2201-23ea-4c01-bc74-bd68eab4caef None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:45.925407 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c97e00-dada-43bc-8666-e362b6a29c8e None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:45.952097 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8a2201-23ea-4c01-bc74-bd68eab4caef None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:45.962801 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee044dbe-8ea9-4352-9d9b-47ae9801cdb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:45.964104 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee044dbe-8ea9-4352-9d9b-47ae9801cdb5 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:45.964396 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee044dbe-8ea9-4352-9d9b-47ae9801cdb5 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:45.964671 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee044dbe-8ea9-4352-9d9b-47ae9801cdb5 None admin] PATH_INFO: `/v3/services` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:45.965296 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee044dbe-8ea9-4352-9d9b-47ae9801cdb5 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:45.965598 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee044dbe-8ea9-4352-9d9b-47ae9801cdb5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:45.967099 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee044dbe-8ea9-4352-9d9b-47ae9801cdb5 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:45.977669 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 494/982] 10.222.0.15 () {56 vars in 1122 bytes} [Sun Sep 11 10:34:45 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:45.978681 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee044dbe-8ea9-4352-9d9b-47ae9801cdb5 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:45.994950 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51036ab3-8c9b-41b8-a62e-f972d584f998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:45.995594 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51036ab3-8c9b-41b8-a62e-f972d584f998 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:45.995872 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51036ab3-8c9b-41b8-a62e-f972d584f998 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:45.996138 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51036ab3-8c9b-41b8-a62e-f972d584f998 None admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:45.996888 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51036ab3-8c9b-41b8-a62e-f972d584f998 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:45.997189 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51036ab3-8c9b-41b8-a62e-f972d584f998 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:45.999070 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51036ab3-8c9b-41b8-a62e-f972d584f998 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:46.004346 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:46.013484 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 489/983] 10.222.0.15 () {56 vars in 1148 bytes} [Sun Sep 11 10:34:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:46.019267 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51036ab3-8c9b-41b8-a62e-f972d584f998 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:46.080190 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:46.090908 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee044dbe-8ea9-4352-9d9b-47ae9801cdb5 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:46.127584 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19f08ba-03d5-41a8-a01b-4d7943416eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:46.128196 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f08ba-03d5-41a8-a01b-4d7943416eb2 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:46.128481 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f08ba-03d5-41a8-a01b-4d7943416eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:46.128758 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f08ba-03d5-41a8-a01b-4d7943416eb2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:46.128968 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51036ab3-8c9b-41b8-a62e-f972d584f998 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:46.129457 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19f08ba-03d5-41a8-a01b-4d7943416eb2 None admin] RBAC: Adding query filter params (interface=admin, service_id=b03c147cdbc94b38ba1e00ae0c3a10ce, region_id=RegionOne) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:46.129767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19f08ba-03d5-41a8-a01b-4d7943416eb2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:46.131192 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19f08ba-03d5-41a8-a01b-4d7943416eb2 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:46.141212 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 495/984] 10.222.0.15 () {56 vars in 1249 bytes} [Sun Sep 11 10:34:45 2022] GET /identity/v3/endpoints?service_id=b03c147cdbc94b38ba1e00ae0c3a10ce&interface=admin®ion_id=RegionOne => generated 193 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:34:46.142227 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19f08ba-03d5-41a8-a01b-4d7943416eb2 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:46.172430 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318a8375-ee75-4e64-88f0-8c3852fd76d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:46.173040 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318a8375-ee75-4e64-88f0-8c3852fd76d5 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:46.173325 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318a8375-ee75-4e64-88f0-8c3852fd76d5 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:46.173599 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318a8375-ee75-4e64-88f0-8c3852fd76d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:46.174325 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318a8375-ee75-4e64-88f0-8c3852fd76d5 None admin] RBAC: Adding query filter params (role.id=55b167894e584c9b8303c60cf3a75a65, scope.project.id=5e1f845f99f04202b83621cb65e0adc2, user.id=bb44e7aef0c7436d93a7233307b5f095) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:46.174622 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318a8375-ee75-4e64-88f0-8c3852fd76d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:46.176078 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318a8375-ee75-4e64-88f0-8c3852fd76d5 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:46.182436 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 490/985] 10.222.0.15 () {56 vars in 1374 bytes} [Sun Sep 11 10:34:46 2022] GET /identity/v3/role_assignments?user.id=bb44e7aef0c7436d93a7233307b5f095&scope.project.id=5e1f845f99f04202b83621cb65e0adc2&role.id=55b167894e584c9b8303c60cf3a75a65 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:46.252829 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:46.266757 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19f08ba-03d5-41a8-a01b-4d7943416eb2 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:46.314314 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29db7222-a22b-421b-9e8b-2b71898cc902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:46.314945 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db7222-a22b-421b-9e8b-2b71898cc902 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:46.315220 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db7222-a22b-421b-9e8b-2b71898cc902 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:46.315515 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db7222-a22b-421b-9e8b-2b71898cc902 None admin] PATH_INFO: `/v3/services` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:46.316125 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29db7222-a22b-421b-9e8b-2b71898cc902 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:46.317533 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29db7222-a22b-421b-9e8b-2b71898cc902 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:46.331814 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 496/986] 10.222.0.15 () {56 vars in 1087 bytes} [Sun Sep 11 10:34:46 2022] GET /identity/v3/services => generated 1497 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:34:46.625589 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e92d76-d47a-4ac1-adeb-f18b31ed6091 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:46.626076 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e92d76-d47a-4ac1-adeb-f18b31ed6091 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:46.626076 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e92d76-d47a-4ac1-adeb-f18b31ed6091 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:46.627405 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 491/987] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:34:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:34:46.633362 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4145ae4-6375-48a5-bb85-fc74505b11df None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:46.633651 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4145ae4-6375-48a5-bb85-fc74505b11df None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:46.633924 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4145ae4-6375-48a5-bb85-fc74505b11df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:46.652087 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-e4145ae4-6375-48a5-bb85-fc74505b11df None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:34:46.653273 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-e4145ae4-6375-48a5-bb85-fc74505b11df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:34:46.723633 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 497/988] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:34:46 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 10:34:46.992506 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e92d76-d47a-4ac1-adeb-f18b31ed6091 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:47.098405 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e92d76-d47a-4ac1-adeb-f18b31ed6091 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:47.121881 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:47.137640 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b9aa00-e1f6-4b5b-8f71-2ca8a412dc12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:47.138257 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b9aa00-e1f6-4b5b-8f71-2ca8a412dc12 None admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:47.138540 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b9aa00-e1f6-4b5b-8f71-2ca8a412dc12 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:47.138814 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b9aa00-e1f6-4b5b-8f71-2ca8a412dc12 None admin] PATH_INFO: `/v3/services` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:47.139496 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b9aa00-e1f6-4b5b-8f71-2ca8a412dc12 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:47.140858 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b9aa00-e1f6-4b5b-8f71-2ca8a412dc12 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:47.159462 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 492/989] 10.222.0.15 () {60 vars in 1141 bytes} [Sun Sep 11 10:34:46 2022] POST /identity/v3/services => generated 242 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:34:48.399806 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5619d6-3644-4ddd-a42e-cb443e489ceb None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:48.400351 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5619d6-3644-4ddd-a42e-cb443e489ceb None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:48.400351 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5619d6-3644-4ddd-a42e-cb443e489ceb None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:48.401771 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 498/990] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:34:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:34:48.407267 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58181aa3-acfd-48d9-a589-5eea7fc6ea81 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:48.407738 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58181aa3-acfd-48d9-a589-5eea7fc6ea81 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:48.407920 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58181aa3-acfd-48d9-a589-5eea7fc6ea81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:48.427731 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-58181aa3-acfd-48d9-a589-5eea7fc6ea81 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:34:48.428929 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-58181aa3-acfd-48d9-a589-5eea7fc6ea81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:34:48.495980 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20074df-4bbe-4d67-aa30-f5365cdf9d7c None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:48.496314 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20074df-4bbe-4d67-aa30-f5365cdf9d7c None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:48.496532 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20074df-4bbe-4d67-aa30-f5365cdf9d7c None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:48.497885 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 499/991] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:34:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 10:34:48.502601 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28efb5f0-9f15-429d-be42-ca9520b1fdc3 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:48.502883 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28efb5f0-9f15-429d-be42-ca9520b1fdc3 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:48.503151 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28efb5f0-9f15-429d-be42-ca9520b1fdc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:48.507076 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 493/992] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:34:48 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 10:34:48.521716 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-28efb5f0-9f15-429d-be42-ca9520b1fdc3 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:34:48.523036 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-28efb5f0-9f15-429d-be42-ca9520b1fdc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:34:48.589299 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 500/993] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:34:48 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 10:34:48.773554 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58181aa3-acfd-48d9-a589-5eea7fc6ea81 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:48.856315 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28efb5f0-9f15-429d-be42-ca9520b1fdc3 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:48.870438 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:48.881522 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58181aa3-acfd-48d9-a589-5eea7fc6ea81 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:48.889986 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:48.921768 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbe3a22-0f7b-4b72-b5e4-267254e7faf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:48.922402 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbe3a22-0f7b-4b72-b5e4-267254e7faf2 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:48.922682 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbe3a22-0f7b-4b72-b5e4-267254e7faf2 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:48.922950 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbe3a22-0f7b-4b72-b5e4-267254e7faf2 None admin] PATH_INFO: `/v3/services/image` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:48.923560 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbe3a22-0f7b-4b72-b5e4-267254e7faf2 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:48.924943 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbe3a22-0f7b-4b72-b5e4-267254e7faf2 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:48.934512 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-ffbe3a22-0f7b-4b72-b5e4-267254e7faf2 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 11 10:34:48.934512 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:48.934512 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 10:34:48.934512 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 10:34:48.934512 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 10:34:48.934512 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 10:34:48.934512 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 10:34:48.934512 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 10:34:48.934512 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 11 10:34:48.934512 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:48.934512 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 10:34:48.934512 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:48.934512 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:48.934512 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:48.934512 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:48.934512 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:48.934512 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:48.935594 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:48.935594 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:48.935594 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:48.935594 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:48.935594 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:48.935594 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:48.935594 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 10:34:48.935594 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 10:34:48.935594 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 10:34:48.935594 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 10:34:48.935594 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:48.935594 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:48.935594 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:48.935594 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:48.935594 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:48.935594 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:48.936456 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:48.936456 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:48.936456 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:48.936456 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:48.936456 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:48.936456 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:48.936456 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:48.936456 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:48.936456 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:48.936456 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:48.936456 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 10:34:48.936456 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 10:34:48.936456 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 11 10:34:48.936456 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:48.936456 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 494/994] 10.222.0.15 () {56 vars in 1105 bytes} [Sun Sep 11 10:34:48 2022] GET /identity/v3/services/image => generated 86 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:48.941261 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbe3a22-0f7b-4b72-b5e4-267254e7faf2 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:48.964691 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28efb5f0-9f15-429d-be42-ca9520b1fdc3 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:49.002009 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7d510f-f052-4237-bec6-9a4c80e7b419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:49.002649 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7d510f-f052-4237-bec6-9a4c80e7b419 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:49.002927 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7d510f-f052-4237-bec6-9a4c80e7b419 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:49.003204 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7d510f-f052-4237-bec6-9a4c80e7b419 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:49.003820 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7d510f-f052-4237-bec6-9a4c80e7b419 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:49.005176 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7d510f-f052-4237-bec6-9a4c80e7b419 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:49.014536 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-2a7d510f-f052-4237-bec6-9a4c80e7b419 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 10:34:49.014536 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:49.014536 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 10:34:49.014536 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 10:34:49.014536 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 10:34:49.014536 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 10:34:49.014536 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 10:34:49.014536 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 10:34:49.014536 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 10:34:49.014536 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:49.014536 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 10:34:49.014536 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:49.014536 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:49.014536 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:49.014536 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:49.014536 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:49.014536 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:49.015608 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:49.015608 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:49.015608 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:49.015608 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:49.015608 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:49.015608 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:49.015608 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 10:34:49.015608 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 10:34:49.015608 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 10:34:49.015608 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 10:34:49.015608 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:49.015608 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:49.015608 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:49.015608 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:49.015608 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:49.015608 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:49.020508 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:49.020508 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:49.020508 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:49.020508 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:49.020508 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:49.020508 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:49.020508 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:49.020508 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:49.020508 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:49.020508 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:49.020508 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 10:34:49.020508 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 10:34:49.020508 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 10:34:49.020508 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:49.020508 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 501/995] 10.222.0.15 () {56 vars in 1126 bytes} [Sun Sep 11 10:34:48 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 10:34:49.022352 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7d510f-f052-4237-bec6-9a4c80e7b419 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:49.056118 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbe3a22-0f7b-4b72-b5e4-267254e7faf2 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:49.099716 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ced7938-a87e-4186-83ec-923049d5cb40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:49.100383 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ced7938-a87e-4186-83ec-923049d5cb40 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:49.100688 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ced7938-a87e-4186-83ec-923049d5cb40 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:49.100983 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ced7938-a87e-4186-83ec-923049d5cb40 None admin] PATH_INFO: `/v3/services` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:49.101532 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ced7938-a87e-4186-83ec-923049d5cb40 None admin] RBAC: Adding query filter params (name=image) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:49.101831 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ced7938-a87e-4186-83ec-923049d5cb40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:49.104237 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ced7938-a87e-4186-83ec-923049d5cb40 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:49.111637 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 495/996] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:34:48 2022] GET /identity/v3/services?name=image => generated 122 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:34:49.116549 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ced7938-a87e-4186-83ec-923049d5cb40 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:49.133884 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:49.159803 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:49.163386 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7d510f-f052-4237-bec6-9a4c80e7b419 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:49.228736 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc9983f-5975-401e-8229-20b28fc6015f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:49.229675 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc9983f-5975-401e-8229-20b28fc6015f None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:49.229813 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc9983f-5975-401e-8229-20b28fc6015f None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:49.230164 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc9983f-5975-401e-8229-20b28fc6015f None admin] PATH_INFO: `/v3/services` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:49.230859 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc9983f-5975-401e-8229-20b28fc6015f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:49.231231 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc9983f-5975-401e-8229-20b28fc6015f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:49.233025 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc9983f-5975-401e-8229-20b28fc6015f None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:49.238507 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 502/997] 10.222.0.15 () {56 vars in 1122 bytes} [Sun Sep 11 10:34:49 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:49.245115 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc9983f-5975-401e-8229-20b28fc6015f None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:49.250884 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ced7938-a87e-4186-83ec-923049d5cb40 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:49.299290 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a602b8-3085-4734-9936-c013895dc755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:49.299950 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a602b8-3085-4734-9936-c013895dc755 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:49.300206 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a602b8-3085-4734-9936-c013895dc755 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:49.300473 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a602b8-3085-4734-9936-c013895dc755 None admin] PATH_INFO: `/v3/services` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:49.301095 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a602b8-3085-4734-9936-c013895dc755 None admin] RBAC: Adding query filter params (type=image) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:49.301396 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a602b8-3085-4734-9936-c013895dc755 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:49.303070 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a602b8-3085-4734-9936-c013895dc755 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:49.309583 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 496/998] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:34:49 2022] GET /identity/v3/services?type=image => generated 122 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:49.366059 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc9983f-5975-401e-8229-20b28fc6015f None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:49.377403 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dffdf0-c632-466a-ba74-c07e73d5662b None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:49.377675 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dffdf0-c632-466a-ba74-c07e73d5662b None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:49.377936 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dffdf0-c632-466a-ba74-c07e73d5662b None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:49.384992 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 497/999] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:34:49 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 10:34:49.386409 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fcf003-901b-4933-8496-533ef582428d None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:49.386699 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fcf003-901b-4933-8496-533ef582428d None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:49.386975 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fcf003-901b-4933-8496-533ef582428d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:49.406063 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044bd15e-9c77-43cd-be27-c2188d498408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:49.406894 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-e4fcf003-901b-4933-8496-533ef582428d None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:34:49.407718 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044bd15e-9c77-43cd-be27-c2188d498408 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:49.408170 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-e4fcf003-901b-4933-8496-533ef582428d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:34:49.408698 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044bd15e-9c77-43cd-be27-c2188d498408 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:49.409191 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044bd15e-9c77-43cd-be27-c2188d498408 None admin] PATH_INFO: `/v3/services` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:49.410037 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044bd15e-9c77-43cd-be27-c2188d498408 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:49.410633 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044bd15e-9c77-43cd-be27-c2188d498408 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:49.413030 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044bd15e-9c77-43cd-be27-c2188d498408 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:49.423117 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 503/1000] 10.222.0.15 () {56 vars in 1122 bytes} [Sun Sep 11 10:34:49 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:34:49.425691 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044bd15e-9c77-43cd-be27-c2188d498408 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:49.430118 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:49.501297 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 498/1001] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:34:49 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 10:34:49.548489 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044bd15e-9c77-43cd-be27-c2188d498408 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:49.582526 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910a6ca9-6096-471f-b024-a5b82a29c1bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:49.583137 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a6ca9-6096-471f-b024-a5b82a29c1bd None admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:49.583432 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a6ca9-6096-471f-b024-a5b82a29c1bd None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:49.584339 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a6ca9-6096-471f-b024-a5b82a29c1bd None admin] PATH_INFO: `/v3/endpoints` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:49.585056 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910a6ca9-6096-471f-b024-a5b82a29c1bd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:49.586420 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910a6ca9-6096-471f-b024-a5b82a29c1bd None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:49.611451 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 504/1002] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:34:49 2022] POST /identity/v3/endpoints => generated 325 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 10:34:49.773758 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fcf003-901b-4933-8496-533ef582428d None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:49.804266 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:49.881741 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fcf003-901b-4933-8496-533ef582428d None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:49.923040 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37efe677-7005-4c2f-b697-d17f9447ac0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:49.924047 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37efe677-7005-4c2f-b697-d17f9447ac0c None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:49.924550 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37efe677-7005-4c2f-b697-d17f9447ac0c None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:49.925012 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37efe677-7005-4c2f-b697-d17f9447ac0c None admin] PATH_INFO: `/v3/services/compute` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:49.925771 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37efe677-7005-4c2f-b697-d17f9447ac0c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:49.927907 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37efe677-7005-4c2f-b697-d17f9447ac0c None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:49.936868 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-37efe677-7005-4c2f-b697-d17f9447ac0c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 11 10:34:49.936868 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:49.936868 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 10:34:49.936868 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 10:34:49.936868 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 10:34:49.936868 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 10:34:49.936868 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 10:34:49.936868 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 10:34:49.936868 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 11 10:34:49.936868 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:49.936868 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 10:34:49.936868 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:49.936868 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:49.936868 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:49.936868 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:49.936868 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:49.936868 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:49.938036 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:49.938036 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:49.938036 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:49.938036 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:49.938036 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:49.938036 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:49.938036 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 10:34:49.938036 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 10:34:49.938036 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 10:34:49.938036 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 10:34:49.938036 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:49.938036 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:49.938036 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:49.938036 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:49.938036 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:49.938036 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:49.939070 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:49.939070 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:49.939070 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:49.939070 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:49.939070 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:49.939070 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:49.939070 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:49.939070 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:49.939070 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:49.939070 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:49.939070 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 10:34:49.939070 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 10:34:49.939070 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 11 10:34:49.939070 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:49.941394 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 499/1003] 10.222.0.15 () {56 vars in 1111 bytes} [Sun Sep 11 10:34:49 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 10:34:49.948127 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910a6ca9-6096-471f-b024-a5b82a29c1bd None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:50.011490 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:50.053549 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910a6ca9-6096-471f-b024-a5b82a29c1bd None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:50.087404 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95589a89-7fd2-4c35-8f39-c3fbd1f046c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:50.088036 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95589a89-7fd2-4c35-8f39-c3fbd1f046c4 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:50.088312 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95589a89-7fd2-4c35-8f39-c3fbd1f046c4 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:50.088581 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95589a89-7fd2-4c35-8f39-c3fbd1f046c4 None admin] PATH_INFO: `/v3/services` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:50.089196 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95589a89-7fd2-4c35-8f39-c3fbd1f046c4 None admin] RBAC: Adding query filter params (name=compute) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:50.089497 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95589a89-7fd2-4c35-8f39-c3fbd1f046c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:50.090852 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95589a89-7fd2-4c35-8f39-c3fbd1f046c4 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:50.096034 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 505/1004] 10.222.0.15 () {56 vars in 1112 bytes} [Sun Sep 11 10:34:49 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:50.102657 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37efe677-7005-4c2f-b697-d17f9447ac0c None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:50.194039 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:50.207632 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37efe677-7005-4c2f-b697-d17f9447ac0c None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:50.243948 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78151d7-a413-4fb2-82be-a9512d914427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:50.244568 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78151d7-a413-4fb2-82be-a9512d914427 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:50.244836 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78151d7-a413-4fb2-82be-a9512d914427 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:50.245105 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78151d7-a413-4fb2-82be-a9512d914427 None admin] PATH_INFO: `/v3/services` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:50.245721 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78151d7-a413-4fb2-82be-a9512d914427 None admin] RBAC: Adding query filter params (type=compute) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:50.246020 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78151d7-a413-4fb2-82be-a9512d914427 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:50.247378 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78151d7-a413-4fb2-82be-a9512d914427 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:50.252407 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 500/1005] 10.222.0.15 () {56 vars in 1112 bytes} [Sun Sep 11 10:34:50 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:50.258582 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95589a89-7fd2-4c35-8f39-c3fbd1f046c4 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:50.371202 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95589a89-7fd2-4c35-8f39-c3fbd1f046c4 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:50.397146 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:50.406198 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bbdfab-08f1-49aa-b5da-3ef91ea7cdc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:50.406793 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bbdfab-08f1-49aa-b5da-3ef91ea7cdc3 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:50.407077 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bbdfab-08f1-49aa-b5da-3ef91ea7cdc3 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:50.407351 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bbdfab-08f1-49aa-b5da-3ef91ea7cdc3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:50.408068 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bbdfab-08f1-49aa-b5da-3ef91ea7cdc3 None admin] RBAC: Adding query filter params (interface=public, service_id=b59ab91b97bb4954a4b3e3f4d1cd6dbf, region_id=RegionOne) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:50.408376 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bbdfab-08f1-49aa-b5da-3ef91ea7cdc3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:50.409737 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bbdfab-08f1-49aa-b5da-3ef91ea7cdc3 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:50.419508 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 506/1006] 10.222.0.15 () {56 vars in 1251 bytes} [Sun Sep 11 10:34:50 2022] GET /identity/v3/endpoints?service_id=b59ab91b97bb4954a4b3e3f4d1cd6dbf&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:34:50.421534 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78151d7-a413-4fb2-82be-a9512d914427 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:50.523588 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78151d7-a413-4fb2-82be-a9512d914427 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:50.561099 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffff02a0-98b6-4da6-8858-5ce2d240906b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:50.561626 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffff02a0-98b6-4da6-8858-5ce2d240906b None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:50.561902 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffff02a0-98b6-4da6-8858-5ce2d240906b None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:50.562177 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffff02a0-98b6-4da6-8858-5ce2d240906b None admin] PATH_INFO: `/v3/services` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:50.562767 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffff02a0-98b6-4da6-8858-5ce2d240906b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:50.564150 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffff02a0-98b6-4da6-8858-5ce2d240906b None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:50.570547 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 501/1007] 10.222.0.15 () {56 vars in 1087 bytes} [Sun Sep 11 10:34:50 2022] GET /identity/v3/services => generated 1727 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:34:51.506909 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ef364-f6c3-4d41-bf0c-2c9a8dae25c0 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:51.507446 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ef364-f6c3-4d41-bf0c-2c9a8dae25c0 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:51.507573 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ef364-f6c3-4d41-bf0c-2c9a8dae25c0 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:51.508965 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 507/1008] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:34:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:34:51.514222 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b241ae49-eb08-4e52-9d35-3cf2270e23ae None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:51.514597 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b241ae49-eb08-4e52-9d35-3cf2270e23ae None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:51.514784 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b241ae49-eb08-4e52-9d35-3cf2270e23ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:51.533618 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-b241ae49-eb08-4e52-9d35-3cf2270e23ae None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:34:51.534810 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-b241ae49-eb08-4e52-9d35-3cf2270e23ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:34:51.547133 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:51.611582 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 502/1009] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:34:51 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 10:34:51.820259 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62e0cd-df57-4e60-aa1a-4c796eaa2014 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:51.820587 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62e0cd-df57-4e60-aa1a-4c796eaa2014 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:51.820809 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62e0cd-df57-4e60-aa1a-4c796eaa2014 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:51.822161 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 508/1010] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:34:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:34:51.827393 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670d1f2e-24c2-4b8e-aa7d-8e5153bb03b4 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:51.827713 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670d1f2e-24c2-4b8e-aa7d-8e5153bb03b4 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:51.827984 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670d1f2e-24c2-4b8e-aa7d-8e5153bb03b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:51.846108 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-670d1f2e-24c2-4b8e-aa7d-8e5153bb03b4 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:34:51.847290 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-670d1f2e-24c2-4b8e-aa7d-8e5153bb03b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:34:51.881967 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d62e0cd-df57-4e60-aa1a-4c796eaa2014 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:51.920685 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 503/1011] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:34:51 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 10:34:51.988392 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d62e0cd-df57-4e60-aa1a-4c796eaa2014 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:52.023219 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8590de-6bf2-4c53-ad9a-10ac0dd967af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:52.023856 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8590de-6bf2-4c53-ad9a-10ac0dd967af None admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:52.024139 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8590de-6bf2-4c53-ad9a-10ac0dd967af None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:52.024415 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8590de-6bf2-4c53-ad9a-10ac0dd967af None admin] PATH_INFO: `/v3/services` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:52.025053 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8590de-6bf2-4c53-ad9a-10ac0dd967af None admin] RBAC: Authorizing `identity:create_service()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:52.026413 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8590de-6bf2-4c53-ad9a-10ac0dd967af None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:52.043544 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 509/1012] 10.222.0.15 () {60 vars in 1141 bytes} [Sun Sep 11 10:34:51 2022] POST /identity/v3/services => generated 242 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 10:34:52.192531 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670d1f2e-24c2-4b8e-aa7d-8e5153bb03b4 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:52.276301 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:52.301757 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670d1f2e-24c2-4b8e-aa7d-8e5153bb03b4 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:52.346244 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63762ef2-9f77-46f4-8c5a-b5abb829824c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:52.346882 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63762ef2-9f77-46f4-8c5a-b5abb829824c None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:52.347161 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63762ef2-9f77-46f4-8c5a-b5abb829824c None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:52.347459 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63762ef2-9f77-46f4-8c5a-b5abb829824c None admin] PATH_INFO: `/v3/domains/default` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:52.348636 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63762ef2-9f77-46f4-8c5a-b5abb829824c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:52.350734 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63762ef2-9f77-46f4-8c5a-b5abb829824c None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:52.352802 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 504/1013] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:34:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:52.358219 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8590de-6bf2-4c53-ad9a-10ac0dd967af None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:52.415324 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:52.466119 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8590de-6bf2-4c53-ad9a-10ac0dd967af None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:52.500576 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0ef4f0-91d0-4e08-8db8-89f966f2cdb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:52.501186 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ef4f0-91d0-4e08-8db8-89f966f2cdb7 None admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:52.501463 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ef4f0-91d0-4e08-8db8-89f966f2cdb7 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:52.501733 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ef4f0-91d0-4e08-8db8-89f966f2cdb7 None admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:52.502380 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0ef4f0-91d0-4e08-8db8-89f966f2cdb7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:52.503828 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0ef4f0-91d0-4e08-8db8-89f966f2cdb7 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:52.535470 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 510/1014] 10.222.0.15 () {60 vars in 1141 bytes} [Sun Sep 11 10:34:52 2022] POST /identity/v3/projects => generated 311 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 10:34:52.765061 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653d8551-1ad1-4e5c-836e-83977d4b8e50 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:52.765560 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653d8551-1ad1-4e5c-836e-83977d4b8e50 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:52.765560 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653d8551-1ad1-4e5c-836e-83977d4b8e50 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:52.766826 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 505/1015] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:34:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:34:52.772726 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59665884-1794-47ef-8c0b-a6b8c8f96fed None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:52.773014 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59665884-1794-47ef-8c0b-a6b8c8f96fed None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:52.773287 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59665884-1794-47ef-8c0b-a6b8c8f96fed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:52.791844 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-59665884-1794-47ef-8c0b-a6b8c8f96fed None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:34:52.793023 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-59665884-1794-47ef-8c0b-a6b8c8f96fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:34:52.861844 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 511/1016] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:34:52 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 10:34:53.131868 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653d8551-1ad1-4e5c-836e-83977d4b8e50 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:53.236296 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653d8551-1ad1-4e5c-836e-83977d4b8e50 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:53.271769 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ed437e-20df-42f7-8800-9a9a846cef28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:53.272644 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed437e-20df-42f7-8800-9a9a846cef28 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:53.273077 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed437e-20df-42f7-8800-9a9a846cef28 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:53.273526 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed437e-20df-42f7-8800-9a9a846cef28 None admin] PATH_INFO: `/v3/services/compute` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:53.274261 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ed437e-20df-42f7-8800-9a9a846cef28 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:53.275806 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ed437e-20df-42f7-8800-9a9a846cef28 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:53.285851 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-a1ed437e-20df-42f7-8800-9a9a846cef28 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 11 10:34:53.285851 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:53.285851 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 10:34:53.285851 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 10:34:53.285851 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 10:34:53.285851 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 10:34:53.285851 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 10:34:53.285851 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 10:34:53.285851 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 11 10:34:53.285851 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:53.285851 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 10:34:53.285851 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:53.285851 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:53.285851 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:53.285851 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:53.285851 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:53.285851 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:53.287712 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:53.287712 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:53.287712 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:53.287712 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:53.287712 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:53.287712 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:53.287712 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 10:34:53.287712 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 10:34:53.287712 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 10:34:53.287712 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 10:34:53.287712 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:53.287712 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:53.287712 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:53.287712 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:53.287712 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:53.287712 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:53.288607 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:53.288607 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:53.288607 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:53.288607 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:53.288607 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:53.288607 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:53.288607 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:53.288607 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:53.288607 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:53.288607 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:53.288607 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 10:34:53.288607 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 10:34:53.288607 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 11 10:34:53.288607 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:53.291612 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 506/1017] 10.222.0.15 () {56 vars in 1111 bytes} [Sun Sep 11 10:34:53 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:53.293525 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59665884-1794-47ef-8c0b-a6b8c8f96fed None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:53.333454 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:53.403874 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59665884-1794-47ef-8c0b-a6b8c8f96fed None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:53.437891 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb76943-3618-4d30-9c7f-4415af926309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:53.438500 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb76943-3618-4d30-9c7f-4415af926309 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:53.438783 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb76943-3618-4d30-9c7f-4415af926309 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:53.439060 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb76943-3618-4d30-9c7f-4415af926309 None admin] PATH_INFO: `/v3/services` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:53.439731 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb76943-3618-4d30-9c7f-4415af926309 None admin] RBAC: Adding query filter params (name=compute) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:53.440038 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb76943-3618-4d30-9c7f-4415af926309 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:53.441399 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb76943-3618-4d30-9c7f-4415af926309 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:53.451912 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 512/1018] 10.222.0.15 () {56 vars in 1112 bytes} [Sun Sep 11 10:34:53 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:53.453067 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ed437e-20df-42f7-8800-9a9a846cef28 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:53.458148 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:53.558182 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ed437e-20df-42f7-8800-9a9a846cef28 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:53.591472 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c54f56a-78a2-4f2f-8b5d-93f98b40cb39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:53.592086 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c54f56a-78a2-4f2f-8b5d-93f98b40cb39 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:53.592367 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c54f56a-78a2-4f2f-8b5d-93f98b40cb39 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:53.592640 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c54f56a-78a2-4f2f-8b5d-93f98b40cb39 None admin] PATH_INFO: `/v3/services` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:53.593265 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c54f56a-78a2-4f2f-8b5d-93f98b40cb39 None admin] RBAC: Adding query filter params (type=compute) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:53.593570 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c54f56a-78a2-4f2f-8b5d-93f98b40cb39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:53.594927 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c54f56a-78a2-4f2f-8b5d-93f98b40cb39 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:53.605269 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 507/1019] 10.222.0.15 () {56 vars in 1112 bytes} [Sun Sep 11 10:34:53 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:53.606184 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb76943-3618-4d30-9c7f-4415af926309 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:53.709688 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:53.714629 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb76943-3618-4d30-9c7f-4415af926309 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:53.756585 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733042c7-e6a4-49a3-92d8-5831e588c5fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:53.757120 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733042c7-e6a4-49a3-92d8-5831e588c5fc None admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:53.757401 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733042c7-e6a4-49a3-92d8-5831e588c5fc None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:53.757675 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733042c7-e6a4-49a3-92d8-5831e588c5fc None admin] PATH_INFO: `/v3/endpoints` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:53.758346 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733042c7-e6a4-49a3-92d8-5831e588c5fc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:53.759757 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733042c7-e6a4-49a3-92d8-5831e588c5fc None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:53.786215 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 513/1020] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:34:53 2022] POST /identity/v3/endpoints => generated 334 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 10:34:54.275099 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb61859-1c73-4c78-9aee-2715e3ac5465 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:54.275680 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb61859-1c73-4c78-9aee-2715e3ac5465 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:54.275680 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb61859-1c73-4c78-9aee-2715e3ac5465 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:54.277153 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 508/1021] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:34:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:34:54.283652 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093ecd1d-9a7d-46ff-a457-245a0db0457c None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:54.283946 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093ecd1d-9a7d-46ff-a457-245a0db0457c None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:54.284225 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093ecd1d-9a7d-46ff-a457-245a0db0457c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:54.302996 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-093ecd1d-9a7d-46ff-a457-245a0db0457c None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:34:54.304211 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-093ecd1d-9a7d-46ff-a457-245a0db0457c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:34:54.382458 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 514/1022] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:34:54 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 10:34:54.654524 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb61859-1c73-4c78-9aee-2715e3ac5465 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:54.726844 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:54.760897 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbc0ae8-abc0-4c24-9a58-26fe9fc9d8f2 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:54.761439 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbc0ae8-abc0-4c24-9a58-26fe9fc9d8f2 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:54.761439 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbc0ae8-abc0-4c24-9a58-26fe9fc9d8f2 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:54.762821 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 515/1023] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:34:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:34:54.767182 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb61859-1c73-4c78-9aee-2715e3ac5465 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:54.768472 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac20b31b-a1a8-4bb0-b5e7-f5b1d62e1bab None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:54.768759 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac20b31b-a1a8-4bb0-b5e7-f5b1d62e1bab None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:54.769035 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac20b31b-a1a8-4bb0-b5e7-f5b1d62e1bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:54.786483 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-ac20b31b-a1a8-4bb0-b5e7-f5b1d62e1bab None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:34:54.788230 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-ac20b31b-a1a8-4bb0-b5e7-f5b1d62e1bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:34:54.810476 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b41680b-a80e-40a9-a9c2-9a6c4b589f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:54.811262 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b41680b-a80e-40a9-a9c2-9a6c4b589f1f None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:54.811573 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b41680b-a80e-40a9-a9c2-9a6c4b589f1f None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:54.811845 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b41680b-a80e-40a9-a9c2-9a6c4b589f1f None admin] PATH_INFO: `/v3/services/image` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:54.812419 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b41680b-a80e-40a9-a9c2-9a6c4b589f1f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:54.813773 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b41680b-a80e-40a9-a9c2-9a6c4b589f1f None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:54.823013 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-9b41680b-a80e-40a9-a9c2-9a6c4b589f1f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 11 10:34:54.823013 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:54.823013 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 10:34:54.823013 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 10:34:54.823013 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 10:34:54.823013 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 10:34:54.823013 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 10:34:54.823013 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 10:34:54.823013 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 11 10:34:54.823013 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:54.823013 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 10:34:54.823013 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:54.823013 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:54.823013 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:54.823013 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:54.823013 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:54.823013 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:54.824163 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:54.824163 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:54.824163 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:54.824163 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:54.824163 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:54.824163 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:54.824163 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 10:34:54.824163 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 10:34:54.824163 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 10:34:54.824163 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 10:34:54.824163 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:54.824163 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:54.824163 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:54.824163 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:54.824163 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:54.824163 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:54.825115 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:54.825115 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:54.825115 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:54.825115 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:54.825115 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:54.825115 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:54.825115 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:54.825115 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:54.825115 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:54.825115 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:54.825115 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 10:34:54.825115 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 10:34:54.825115 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 11 10:34:54.825115 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:54.825115 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 509/1024] 10.222.0.15 () {56 vars in 1105 bytes} [Sun Sep 11 10:34:54 2022] GET /identity/v3/services/image => generated 86 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:54.829998 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b41680b-a80e-40a9-a9c2-9a6c4b589f1f None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:54.863608 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 516/1025] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:34:54 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 10:34:54.932463 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b41680b-a80e-40a9-a9c2-9a6c4b589f1f None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:54.960961 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:54.966936 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907d8903-fe43-4675-8c06-8921bce37d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:54.967571 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d8903-fe43-4675-8c06-8921bce37d5d None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:54.967854 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d8903-fe43-4675-8c06-8921bce37d5d None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:54.968122 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d8903-fe43-4675-8c06-8921bce37d5d None admin] PATH_INFO: `/v3/services` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:54.968739 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907d8903-fe43-4675-8c06-8921bce37d5d None admin] RBAC: Adding query filter params (name=image) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:54.969041 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907d8903-fe43-4675-8c06-8921bce37d5d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:54.970401 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907d8903-fe43-4675-8c06-8921bce37d5d None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:54.980968 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 510/1026] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:34:54 2022] GET /identity/v3/services?name=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:34:54.982056 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac20b31b-a1a8-4bb0-b5e7-f5b1d62e1bab None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:54.994473 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:55.088019 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac20b31b-a1a8-4bb0-b5e7-f5b1d62e1bab None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:55.123793 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658a4e83-2833-4ae1-941b-61aa2a9321d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:55.124417 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a4e83-2833-4ae1-941b-61aa2a9321d5 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:55.124699 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a4e83-2833-4ae1-941b-61aa2a9321d5 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:55.124971 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a4e83-2833-4ae1-941b-61aa2a9321d5 None admin] PATH_INFO: `/v3/services` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:55.125601 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658a4e83-2833-4ae1-941b-61aa2a9321d5 None admin] RBAC: Adding query filter params (type=image) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:55.125905 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658a4e83-2833-4ae1-941b-61aa2a9321d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:55.127268 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658a4e83-2833-4ae1-941b-61aa2a9321d5 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:55.137407 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 517/1027] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:34:54 2022] GET /identity/v3/services?type=image => generated 350 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:55.138376 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907d8903-fe43-4675-8c06-8921bce37d5d None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:55.138798 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658a4e83-2833-4ae1-941b-61aa2a9321d5 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:55.224643 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:55.250283 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658a4e83-2833-4ae1-941b-61aa2a9321d5 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:55.251130 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907d8903-fe43-4675-8c06-8921bce37d5d None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:55.287476 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33cc643-8d43-4e39-a5a7-e3170fdbb2e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:55.288110 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33cc643-8d43-4e39-a5a7-e3170fdbb2e0 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:55.288431 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33cc643-8d43-4e39-a5a7-e3170fdbb2e0 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:55.288652 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33cc643-8d43-4e39-a5a7-e3170fdbb2e0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:55.289738 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33cc643-8d43-4e39-a5a7-e3170fdbb2e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:55.291173 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33cc643-8d43-4e39-a5a7-e3170fdbb2e0 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:55.295463 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 518/1028] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:34:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:34:55.295600 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe63402-da50-41cd-b550-8955e233730c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:55.296026 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe63402-da50-41cd-b550-8955e233730c None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:55.296179 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe63402-da50-41cd-b550-8955e233730c None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:55.296430 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe63402-da50-41cd-b550-8955e233730c None admin] PATH_INFO: `/v3/endpoints` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:55.297127 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe63402-da50-41cd-b550-8955e233730c None admin] RBAC: Adding query filter params (interface=public, service_id=f79ec1844bd24437a8df659c57d2ed82, region_id=RegionOne) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:55.297711 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe63402-da50-41cd-b550-8955e233730c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:55.298425 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33cc643-8d43-4e39-a5a7-e3170fdbb2e0 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:55.299069 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe63402-da50-41cd-b550-8955e233730c None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:55.307453 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 511/1029] 10.222.0.15 () {56 vars in 1251 bytes} [Sun Sep 11 10:34:55 2022] GET /identity/v3/endpoints?service_id=f79ec1844bd24437a8df659c57d2ed82&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:34:55.311641 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe63402-da50-41cd-b550-8955e233730c None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:55.381608 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:55.408601 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33cc643-8d43-4e39-a5a7-e3170fdbb2e0 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:55.425992 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe63402-da50-41cd-b550-8955e233730c None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:55.444370 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cd5e97-34d4-4d3a-86fc-c46580d385e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:55.444990 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cd5e97-34d4-4d3a-86fc-c46580d385e6 None admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:55.445273 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cd5e97-34d4-4d3a-86fc-c46580d385e6 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:55.445548 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cd5e97-34d4-4d3a-86fc-c46580d385e6 None admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:55.446197 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cd5e97-34d4-4d3a-86fc-c46580d385e6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:55.447670 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cd5e97-34d4-4d3a-86fc-c46580d385e6 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:55.464071 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df65bbf0-1436-4dfd-99a3-af71cf5a8e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:55.464776 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df65bbf0-1436-4dfd-99a3-af71cf5a8e36 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:55.465096 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df65bbf0-1436-4dfd-99a3-af71cf5a8e36 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:55.465372 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df65bbf0-1436-4dfd-99a3-af71cf5a8e36 None admin] PATH_INFO: `/v3/services` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:55.465509 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:55.466059 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df65bbf0-1436-4dfd-99a3-af71cf5a8e36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:55.467580 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df65bbf0-1436-4dfd-99a3-af71cf5a8e36 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:55.483569 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 512/1030] 10.222.0.15 () {56 vars in 1087 bytes} [Sun Sep 11 10:34:55 2022] GET /identity/v3/services => generated 1957 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 10:34:55.483774 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 519/1031] 10.222.0.15 () {60 vars in 1132 bytes} [Sun Sep 11 10:34:55 2022] POST /identity/v3/users => generated 285 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 10:34:56.061082 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee00c8-7450-452a-99de-186f39ef2480 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:56.061460 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee00c8-7450-452a-99de-186f39ef2480 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:56.061654 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee00c8-7450-452a-99de-186f39ef2480 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:56.063129 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 513/1032] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:34:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:34:56.068471 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e183ab-2b13-44bc-ba6d-49ca4ba2b88c None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:56.068922 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e183ab-2b13-44bc-ba6d-49ca4ba2b88c None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:56.069101 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e183ab-2b13-44bc-ba6d-49ca4ba2b88c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:56.087460 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-96e183ab-2b13-44bc-ba6d-49ca4ba2b88c None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:34:56.088642 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-96e183ab-2b13-44bc-ba6d-49ca4ba2b88c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:34:56.158706 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 520/1033] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:34:56 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 10:34:56.434379 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ee00c8-7450-452a-99de-186f39ef2480 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:56.545154 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ee00c8-7450-452a-99de-186f39ef2480 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:56.569363 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:56.580879 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a763ec5-f894-4122-94ce-c1b5acbf5fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:56.581514 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a763ec5-f894-4122-94ce-c1b5acbf5fd9 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:56.581796 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a763ec5-f894-4122-94ce-c1b5acbf5fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:56.582069 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a763ec5-f894-4122-94ce-c1b5acbf5fd9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:56.592125 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a763ec5-f894-4122-94ce-c1b5acbf5fd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:34:56.596834 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-7a763ec5-f894-4122-94ce-c1b5acbf5fd9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 10:34:56.596834 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:56.596834 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:56.596834 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:56.596834 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:56.596834 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:56.596834 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:56.596834 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:56.596834 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:56.596834 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:56.596834 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:56.596834 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:56.596834 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:34:56.596834 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:34:56.596834 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:34:56.596834 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:34:56.597759 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:34:56.597759 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:34:56.597759 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:34:56.597759 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:34:56.597759 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:56.597759 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:56.597759 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:56.597759 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:56.597759 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:56.597759 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:56.597759 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:56.597759 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:56.597759 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:56.597759 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:56.597759 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:56.597759 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:56.598604 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:56.598604 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:56.598604 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:56.598604 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:56.598604 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:34:56.598604 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:34:56.598604 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:34:56.598604 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:34:56.598604 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:34:56.598604 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:34:56.598604 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 10:34:56.598604 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:56.603262 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 514/1034] 10.222.0.15 () {56 vars in 1120 bytes} [Sun Sep 11 10:34:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:56.604550 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e183ab-2b13-44bc-ba6d-49ca4ba2b88c None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:56.707498 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e183ab-2b13-44bc-ba6d-49ca4ba2b88c None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:56.735060 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:56.744024 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d986a115-93b5-47e0-814c-3b59b3708212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:56.744634 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d986a115-93b5-47e0-814c-3b59b3708212 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:56.744911 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d986a115-93b5-47e0-814c-3b59b3708212 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:56.745243 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d986a115-93b5-47e0-814c-3b59b3708212 None admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:56.745903 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d986a115-93b5-47e0-814c-3b59b3708212 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:56.746204 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d986a115-93b5-47e0-814c-3b59b3708212 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:56.747585 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d986a115-93b5-47e0-814c-3b59b3708212 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:56.755981 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 521/1035] 10.222.0.15 () {56 vars in 1115 bytes} [Sun Sep 11 10:34:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:56.761427 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a763ec5-f894-4122-94ce-c1b5acbf5fd9 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:56.866736 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a763ec5-f894-4122-94ce-c1b5acbf5fd9 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:56.900695 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016f45a5-edae-4e10-8c6b-5f13e794ab1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:56.901315 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016f45a5-edae-4e10-8c6b-5f13e794ab1d None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:56.901589 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016f45a5-edae-4e10-8c6b-5f13e794ab1d None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:56.901852 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016f45a5-edae-4e10-8c6b-5f13e794ab1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:56.912076 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016f45a5-edae-4e10-8c6b-5f13e794ab1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:56.913849 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016f45a5-edae-4e10-8c6b-5f13e794ab1d None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:56.928705 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-016f45a5-edae-4e10-8c6b-5f13e794ab1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:56.928705 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:56.928705 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:56.928705 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:56.928705 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:56.928705 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:56.928705 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:56.928705 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:56.928705 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:56.928705 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:56.928705 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:56.928705 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:56.928705 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:34:56.928705 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:34:56.928705 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:34:56.928705 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:34:56.929635 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:56.929635 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:56.929635 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:56.929635 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:56.929635 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:56.929635 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:56.929635 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:56.929635 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:56.929635 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:56.929635 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:56.929635 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:56.929635 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:56.929635 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:56.929635 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:56.929635 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:56.929635 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:56.929635 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:34:56.930530 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:34:56.930530 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:56.930530 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:56.930530 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 515/1036] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:34:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:56.936846 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d986a115-93b5-47e0-814c-3b59b3708212 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:57.038192 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d986a115-93b5-47e0-814c-3b59b3708212 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:57.071486 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a80f62-e283-48df-af57-43dedaa04f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:57.072153 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a80f62-e283-48df-af57-43dedaa04f5a None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:57.072447 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a80f62-e283-48df-af57-43dedaa04f5a None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:57.072609 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a80f62-e283-48df-af57-43dedaa04f5a None admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:57.073238 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a80f62-e283-48df-af57-43dedaa04f5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:57.073537 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a80f62-e283-48df-af57-43dedaa04f5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:57.074886 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a80f62-e283-48df-af57-43dedaa04f5a None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:57.087578 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 522/1037] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:34:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:57.092266 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016f45a5-edae-4e10-8c6b-5f13e794ab1d None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:57.131852 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:57.201254 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016f45a5-edae-4e10-8c6b-5f13e794ab1d None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:57.235620 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316da133-63c1-417a-9b7b-3d8b00ab07d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:57.236252 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316da133-63c1-417a-9b7b-3d8b00ab07d5 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:57.236525 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316da133-63c1-417a-9b7b-3d8b00ab07d5 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:57.236791 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316da133-63c1-417a-9b7b-3d8b00ab07d5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:57.244428 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316da133-63c1-417a-9b7b-3d8b00ab07d5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:57.245846 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316da133-63c1-417a-9b7b-3d8b00ab07d5 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:57.258134 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-316da133-63c1-417a-9b7b-3d8b00ab07d5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 11 10:34:57.258134 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:57.258134 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:57.258134 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:57.258134 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:57.258134 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:57.258134 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:57.258134 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:57.258134 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:57.258134 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:57.258134 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:57.258134 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:57.258134 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 10:34:57.258134 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 10:34:57.258134 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 10:34:57.258134 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 10:34:57.259153 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:57.259153 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:57.259153 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 10:34:57.259153 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:34:57.259153 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 10:34:57.259153 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:34:57.259153 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:57.259153 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:57.259153 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:57.259153 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:57.259153 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:57.259153 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:57.259153 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:57.259153 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:57.259153 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:57.259153 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:57.259153 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:57.259153 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:57.264952 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:57.264952 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:57.264952 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 10:34:57.264952 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 10:34:57.264952 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 10:34:57.264952 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 10:34:57.264952 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 10:34:57.264952 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 10:34:57.264952 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 11 10:34:57.264952 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:57.264952 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 516/1038] 10.222.0.15 () {56 vars in 1093 bytes} [Sun Sep 11 10:34:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:57.267125 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a80f62-e283-48df-af57-43dedaa04f5a None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:57.326790 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:57.379020 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a80f62-e283-48df-af57-43dedaa04f5a None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:57.414473 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab3f0f9-08d7-4257-9403-c4fc5b9caf17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:57.415094 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab3f0f9-08d7-4257-9403-c4fc5b9caf17 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:57.415377 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab3f0f9-08d7-4257-9403-c4fc5b9caf17 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:57.415682 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab3f0f9-08d7-4257-9403-c4fc5b9caf17 None admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:57.416407 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab3f0f9-08d7-4257-9403-c4fc5b9caf17 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:57.416714 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab3f0f9-08d7-4257-9403-c4fc5b9caf17 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:57.418187 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab3f0f9-08d7-4257-9403-c4fc5b9caf17 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:57.440514 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 523/1039] 10.222.0.15 () {56 vars in 1133 bytes} [Sun Sep 11 10:34:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:57.442088 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316da133-63c1-417a-9b7b-3d8b00ab07d5 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:57.545600 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:57.553658 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316da133-63c1-417a-9b7b-3d8b00ab07d5 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:57.591497 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8c75ef-6b2d-4ceb-b5ac-f4db0a61d31c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:57.592039 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8c75ef-6b2d-4ceb-b5ac-f4db0a61d31c None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:57.592321 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8c75ef-6b2d-4ceb-b5ac-f4db0a61d31c None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:57.592591 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8c75ef-6b2d-4ceb-b5ac-f4db0a61d31c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:57.666113 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8c75ef-6b2d-4ceb-b5ac-f4db0a61d31c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:57.668021 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8c75ef-6b2d-4ceb-b5ac-f4db0a61d31c None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:57.668730 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d9015-9c2f-4c46-bcf6-0fa85f7e6211 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:57.669022 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d9015-9c2f-4c46-bcf6-0fa85f7e6211 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:57.669298 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d9015-9c2f-4c46-bcf6-0fa85f7e6211 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:57.670862 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 524/1040] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:34:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:34:57.675607 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ff6f9b-1c67-41d4-8266-b64151c30977 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:57.675894 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ff6f9b-1c67-41d4-8266-b64151c30977 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:57.676171 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ff6f9b-1c67-41d4-8266-b64151c30977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:57.694919 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-20ff6f9b-1c67-41d4-8266-b64151c30977 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:34:57.696580 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-20ff6f9b-1c67-41d4-8266-b64151c30977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:34:57.766054 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 525/1041] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:34:57 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 10:34:57.767487 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd4d89f-8c6c-4a94-883e-ea9755bf52d8 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:57.767778 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd4d89f-8c6c-4a94-883e-ea9755bf52d8 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:57.768052 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd4d89f-8c6c-4a94-883e-ea9755bf52d8 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:57.769429 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 526/1042] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:34:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:34:57.775456 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083cbdb7-39b2-4336-83af-87692df99fae None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:57.775738 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083cbdb7-39b2-4336-83af-87692df99fae None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:57.776021 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083cbdb7-39b2-4336-83af-87692df99fae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:57.781502 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:57.789174 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 517/1043] 10.222.0.15 () {58 vars in 1282 bytes} [Sun Sep 11 10:34:57 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 11 10:34:57.794581 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8c75ef-6b2d-4ceb-b5ac-f4db0a61d31c None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:57.799307 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-083cbdb7-39b2-4336-83af-87692df99fae None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:34:57.800531 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-083cbdb7-39b2-4336-83af-87692df99fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:34:57.830751 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:57.873741 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 527/1044] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:34:57 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 10:34:57.898496 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8c75ef-6b2d-4ceb-b5ac-f4db0a61d31c None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:57.932014 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ba0ac6-4d6b-4901-9ced-65f632663c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:57.932946 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ba0ac6-4d6b-4901-9ced-65f632663c2d None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:57.933418 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ba0ac6-4d6b-4901-9ced-65f632663c2d None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:57.933886 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ba0ac6-4d6b-4901-9ced-65f632663c2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:57.944751 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ba0ac6-4d6b-4901-9ced-65f632663c2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:57.946393 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ba0ac6-4d6b-4901-9ced-65f632663c2d None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:57.961822 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-24ba0ac6-4d6b-4901-9ced-65f632663c2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:57.961822 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:57.961822 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:57.961822 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:57.961822 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:57.961822 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:57.961822 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:57.961822 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:57.961822 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:57.961822 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:57.961822 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:57.961822 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:57.961822 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:34:57.961822 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:34:57.961822 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:34:57.961822 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:34:57.962854 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:57.962854 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:57.962854 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:57.962854 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:57.962854 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:57.962854 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:57.962854 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:57.962854 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:57.962854 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:57.962854 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:57.962854 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:57.962854 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:57.962854 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:57.962854 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:57.962854 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:57.962854 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:57.962854 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:34:57.963895 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:34:57.963895 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:34:57.963895 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:57.964879 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 518/1045] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:34:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 10:34:57.970310 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083cbdb7-39b2-4336-83af-87692df99fae None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:58.040186 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ba0ac6-4d6b-4901-9ced-65f632663c2d None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:58.074747 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083cbdb7-39b2-4336-83af-87692df99fae None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:58.100848 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:58.118614 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d258f56e-9f51-4964-acc2-5399b9f64471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:58.119554 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d258f56e-9f51-4964-acc2-5399b9f64471 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:58.119839 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d258f56e-9f51-4964-acc2-5399b9f64471 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:58.120118 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d258f56e-9f51-4964-acc2-5399b9f64471 None admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:58.120758 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d258f56e-9f51-4964-acc2-5399b9f64471 None admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:58.121063 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d258f56e-9f51-4964-acc2-5399b9f64471 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:58.122429 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d258f56e-9f51-4964-acc2-5399b9f64471 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:58.134296 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 528/1046] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:34:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:58.139658 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d258f56e-9f51-4964-acc2-5399b9f64471 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:58.168777 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ba0ac6-4d6b-4901-9ced-65f632663c2d None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:58.169561 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:58.202757 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9a64b2-2e99-4ebe-b09b-c2231c9f1cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:58.203395 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a64b2-2e99-4ebe-b09b-c2231c9f1cf9 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:58.203845 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a64b2-2e99-4ebe-b09b-c2231c9f1cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:58.204111 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a64b2-2e99-4ebe-b09b-c2231c9f1cf9 None admin] PATH_INFO: `/v3/services/image` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:58.204703 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9a64b2-2e99-4ebe-b09b-c2231c9f1cf9 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:58.206267 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9a64b2-2e99-4ebe-b09b-c2231c9f1cf9 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:58.215785 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-ac9a64b2-2e99-4ebe-b09b-c2231c9f1cf9 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 11 10:34:58.215785 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:58.215785 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 10:34:58.215785 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 10:34:58.215785 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 10:34:58.215785 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 10:34:58.215785 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 10:34:58.215785 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 10:34:58.215785 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 11 10:34:58.215785 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:58.215785 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 10:34:58.215785 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:58.215785 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:58.215785 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:58.215785 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:58.215785 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:58.215785 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:58.217055 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:58.217055 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:58.217055 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:58.217055 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:58.217055 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:58.217055 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:58.217055 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 10:34:58.217055 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 10:34:58.217055 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 10:34:58.217055 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 10:34:58.217055 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:58.217055 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:58.217055 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:58.217055 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:58.217055 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:58.217055 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:58.217919 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:58.217919 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:58.217919 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:58.217919 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:58.217919 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:58.217919 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:58.217919 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:58.217919 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:58.217919 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:58.217919 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:58.217919 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 10:34:58.217919 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 10:34:58.217919 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 11 10:34:58.217919 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:58.219167 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 519/1047] 10.222.0.15 () {56 vars in 1105 bytes} [Sun Sep 11 10:34:58 2022] GET /identity/v3/services/image => generated 86 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 10:34:58.220178 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9a64b2-2e99-4ebe-b09b-c2231c9f1cf9 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:58.246984 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d258f56e-9f51-4964-acc2-5399b9f64471 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:58.291129 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab629f93-6ccf-4394-8728-8be058997090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:58.291804 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab629f93-6ccf-4394-8728-8be058997090 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:58.292083 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab629f93-6ccf-4394-8728-8be058997090 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:58.292350 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab629f93-6ccf-4394-8728-8be058997090 None admin] PATH_INFO: `/v3/projects/service` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:58.300385 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab629f93-6ccf-4394-8728-8be058997090 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:58.302391 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab629f93-6ccf-4394-8728-8be058997090 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:58.314656 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-ab629f93-6ccf-4394-8728-8be058997090 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:34:58.314656 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:58.314656 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:58.314656 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:58.314656 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:58.314656 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:58.314656 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:58.314656 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:58.314656 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:58.314656 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:58.314656 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:58.314656 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:58.314656 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 10:34:58.314656 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 10:34:58.314656 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 10:34:58.315694 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 10:34:58.315694 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:58.315694 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:58.315694 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:58.315694 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:58.315694 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:58.315694 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:58.315694 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:58.315694 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:58.315694 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:58.315694 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:58.315694 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:58.315694 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:58.315694 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:58.315694 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:58.315694 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:58.315694 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:58.318544 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 10:34:58.318544 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 10:34:58.318544 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 10:34:58.318544 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 10:34:58.318544 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 10:34:58.318544 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 10:34:58.318544 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:34:58.318544 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:34:58.318544 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 529/1048] 10.222.0.15 () {56 vars in 1111 bytes} [Sun Sep 11 10:34:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 10:34:58.318544 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab629f93-6ccf-4394-8728-8be058997090 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:58.331764 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9a64b2-2e99-4ebe-b09b-c2231c9f1cf9 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:58.347825 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:58.376313 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb3d2e7-b6e7-42aa-9af9-e4f800827853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:58.377596 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb3d2e7-b6e7-42aa-9af9-e4f800827853 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:58.377876 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb3d2e7-b6e7-42aa-9af9-e4f800827853 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:58.378191 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb3d2e7-b6e7-42aa-9af9-e4f800827853 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:58.393153 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb3d2e7-b6e7-42aa-9af9-e4f800827853 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:34:58.399260 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-acb3d2e7-b6e7-42aa-9af9-e4f800827853 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 10:34:58.399260 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:34:58.399260 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:34:58.399260 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:34:58.399260 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:34:58.399260 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:34:58.399260 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:34:58.399260 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:34:58.399260 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:34:58.399260 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:34:58.399260 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:34:58.399260 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:34:58.399260 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:34:58.399260 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:34:58.399260 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:34:58.399260 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:34:58.400696 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:34:58.400696 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:34:58.400696 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:34:58.400696 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:34:58.400696 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:34:58.400696 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:34:58.400696 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:34:58.400696 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:34:58.400696 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:34:58.400696 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:34:58.400696 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:34:58.400696 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:34:58.400696 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:34:58.400696 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:34:58.400696 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:34:58.400696 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:34:58.401584 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:34:58.401584 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:34:58.401584 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:34:58.401584 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:34:58.401584 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:34:58.401584 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:34:58.401584 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:34:58.401584 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:34:58.401584 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:34:58.401584 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:34:58.401584 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 10:34:58.401584 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:34:58.401584 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 520/1049] 10.222.0.15 () {56 vars in 1096 bytes} [Sun Sep 11 10:34:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:34:58.401584 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb3d2e7-b6e7-42aa-9af9-e4f800827853 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:58.429493 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:58.443367 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab629f93-6ccf-4394-8728-8be058997090 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:58.484422 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e626e8-8163-4607-bdd0-9a32c1fcabbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:58.485045 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e626e8-8163-4607-bdd0-9a32c1fcabbf None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:58.485325 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e626e8-8163-4607-bdd0-9a32c1fcabbf None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:58.485601 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e626e8-8163-4607-bdd0-9a32c1fcabbf None admin] PATH_INFO: `/v3/services` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:58.486235 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e626e8-8163-4607-bdd0-9a32c1fcabbf None admin] RBAC: Adding query filter params (name=image) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:58.486537 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e626e8-8163-4607-bdd0-9a32c1fcabbf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:58.487944 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e626e8-8163-4607-bdd0-9a32c1fcabbf None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:58.494246 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 530/1050] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:34:58 2022] GET /identity/v3/services?name=image => generated 122 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:58.495086 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e626e8-8163-4607-bdd0-9a32c1fcabbf None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:58.511973 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb3d2e7-b6e7-42aa-9af9-e4f800827853 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:58.553911 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f3f726-d8a7-4253-b360-cbf1b0262aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:58.554526 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f3f726-d8a7-4253-b360-cbf1b0262aa6 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:58.554800 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f3f726-d8a7-4253-b360-cbf1b0262aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:58.555071 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f3f726-d8a7-4253-b360-cbf1b0262aa6 None admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:58.555795 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f3f726-d8a7-4253-b360-cbf1b0262aa6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:58.556104 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f3f726-d8a7-4253-b360-cbf1b0262aa6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:58.557544 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f3f726-d8a7-4253-b360-cbf1b0262aa6 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:58.570949 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 521/1051] 10.222.0.15 () {56 vars in 1148 bytes} [Sun Sep 11 10:34:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:58.571848 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f3f726-d8a7-4253-b360-cbf1b0262aa6 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:58.606883 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:58.611298 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e626e8-8163-4607-bdd0-9a32c1fcabbf None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:58.654727 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12e600c-2a7c-436a-83d1-d191b71639b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:58.655741 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e600c-2a7c-436a-83d1-d191b71639b8 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:58.655741 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e600c-2a7c-436a-83d1-d191b71639b8 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:58.656492 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e600c-2a7c-436a-83d1-d191b71639b8 None admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:58.656908 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12e600c-2a7c-436a-83d1-d191b71639b8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:58.657392 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12e600c-2a7c-436a-83d1-d191b71639b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:58.658989 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12e600c-2a7c-436a-83d1-d191b71639b8 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:58.670665 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 531/1052] 10.222.0.15 () {56 vars in 1099 bytes} [Sun Sep 11 10:34:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:58.671200 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12e600c-2a7c-436a-83d1-d191b71639b8 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:58.684855 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f3f726-d8a7-4253-b360-cbf1b0262aa6 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:58.694434 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:58.722142 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8d06bb-f1fe-4a25-848b-38df8c079bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:58.722753 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d06bb-f1fe-4a25-848b-38df8c079bb7 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:58.723031 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d06bb-f1fe-4a25-848b-38df8c079bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:58.723299 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d06bb-f1fe-4a25-848b-38df8c079bb7 None admin] PATH_INFO: `/v3/services` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:58.724003 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8d06bb-f1fe-4a25-848b-38df8c079bb7 None admin] RBAC: Adding query filter params (type=image) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:58.724304 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8d06bb-f1fe-4a25-848b-38df8c079bb7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:58.725674 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8d06bb-f1fe-4a25-848b-38df8c079bb7 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:58.730820 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 522/1053] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:34:58 2022] GET /identity/v3/services?type=image => generated 350 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:58.732381 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8d06bb-f1fe-4a25-848b-38df8c079bb7 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:58.801013 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12e600c-2a7c-436a-83d1-d191b71639b8 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:58.842093 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab576f98-e325-475a-8249-90631b65ed27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:58.842711 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab576f98-e325-475a-8249-90631b65ed27 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:58.842864 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:58.842995 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab576f98-e325-475a-8249-90631b65ed27 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:58.843496 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab576f98-e325-475a-8249-90631b65ed27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:58.844856 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab576f98-e325-475a-8249-90631b65ed27 None admin] RBAC: Adding query filter params (role.id=55b167894e584c9b8303c60cf3a75a65, scope.project.id=5e1f845f99f04202b83621cb65e0adc2, user.id=2bd5356829b6454993bab9bad2f79fed) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:58.845427 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab576f98-e325-475a-8249-90631b65ed27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:58.848119 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab576f98-e325-475a-8249-90631b65ed27 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:58.849575 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8d06bb-f1fe-4a25-848b-38df8c079bb7 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:58.854292 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 532/1054] 10.222.0.15 () {56 vars in 1374 bytes} [Sun Sep 11 10:34:58 2022] GET /identity/v3/role_assignments?user.id=2bd5356829b6454993bab9bad2f79fed&scope.project.id=5e1f845f99f04202b83621cb65e0adc2&role.id=55b167894e584c9b8303c60cf3a75a65 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:58.855081 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab576f98-e325-475a-8249-90631b65ed27 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:58.896289 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71014e5-f99c-4a96-b39e-06487b1db077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:58.896947 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71014e5-f99c-4a96-b39e-06487b1db077 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:58.897221 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71014e5-f99c-4a96-b39e-06487b1db077 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:58.897497 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71014e5-f99c-4a96-b39e-06487b1db077 None admin] PATH_INFO: `/v3/users/2bb7a19cf45d403ba0f5f43d0a43c152` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:58.918416 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71014e5-f99c-4a96-b39e-06487b1db077 None admin] RBAC: Authorizing `identity:get_user(user_id=2bb7a19cf45d403ba0f5f43d0a43c152)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:58.919871 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71014e5-f99c-4a96-b39e-06487b1db077 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:58.921791 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 523/1055] 10.222.0.15 () {56 vars in 1177 bytes} [Sun Sep 11 10:34:58 2022] GET /identity/v3/users/2bb7a19cf45d403ba0f5f43d0a43c152 => generated 285 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:58.928717 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71014e5-f99c-4a96-b39e-06487b1db077 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:58.954012 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:58.980094 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab576f98-e325-475a-8249-90631b65ed27 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:59.028811 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f577dad5-6e71-4173-80de-7a7bfe41bb17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:59.030274 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f577dad5-6e71-4173-80de-7a7bfe41bb17 None admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:59.030404 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f577dad5-6e71-4173-80de-7a7bfe41bb17 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:59.030812 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f577dad5-6e71-4173-80de-7a7bfe41bb17 None admin] PATH_INFO: `/v3/endpoints` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:59.032070 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f577dad5-6e71-4173-80de-7a7bfe41bb17 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:59.034684 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f577dad5-6e71-4173-80de-7a7bfe41bb17 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:59.067138 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 533/1056] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:34:58 2022] POST /identity/v3/endpoints => generated 327 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:34:59.067375 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71014e5-f99c-4a96-b39e-06487b1db077 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:59.114417 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5bf2dc-793d-479a-9c7c-b3f71b87664d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:59.115891 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5bf2dc-793d-479a-9c7c-b3f71b87664d None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:59.116370 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5bf2dc-793d-479a-9c7c-b3f71b87664d None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:59.116855 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5bf2dc-793d-479a-9c7c-b3f71b87664d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:59.145591 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:59.210858 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5bf2dc-793d-479a-9c7c-b3f71b87664d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:59.212265 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5bf2dc-793d-479a-9c7c-b3f71b87664d None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:59.330254 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 524/1057] 10.222.0.15 () {58 vars in 1282 bytes} [Sun Sep 11 10:34:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 10:34:59.337018 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f577dad5-6e71-4173-80de-7a7bfe41bb17 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:59.442521 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f577dad5-6e71-4173-80de-7a7bfe41bb17 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:59.479291 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a7fb53-3380-4ab3-8446-fa9d266e9c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:59.480194 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a7fb53-3380-4ab3-8446-fa9d266e9c4d None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:59.480617 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a7fb53-3380-4ab3-8446-fa9d266e9c4d None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:59.481056 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a7fb53-3380-4ab3-8446-fa9d266e9c4d None admin] PATH_INFO: `/v3/projects/4117990fba5243fcafb57c9a4a5c16d9` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:59.482306 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a7fb53-3380-4ab3-8446-fa9d266e9c4d None admin] RBAC: Authorizing `identity:get_project(project_id=4117990fba5243fcafb57c9a4a5c16d9)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:59.483919 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a7fb53-3380-4ab3-8446-fa9d266e9c4d None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:59.486080 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 534/1058] 10.222.0.15 () {56 vars in 1186 bytes} [Sun Sep 11 10:34:59 2022] GET /identity/v3/projects/4117990fba5243fcafb57c9a4a5c16d9 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:34:59.491895 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5bf2dc-793d-479a-9c7c-b3f71b87664d None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:34:59.545760 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:34:59.600798 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5bf2dc-793d-479a-9c7c-b3f71b87664d None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:34:59.636049 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7484a04f-38df-4ab4-a13f-e49f604b2726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:34:59.636655 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7484a04f-38df-4ab4-a13f-e49f604b2726 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:34:59.636928 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7484a04f-38df-4ab4-a13f-e49f604b2726 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:34:59.637192 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7484a04f-38df-4ab4-a13f-e49f604b2726 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:34:59.637903 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7484a04f-38df-4ab4-a13f-e49f604b2726 None admin] RBAC: Adding query filter params (role.id=2cb7d73f0ea8414296367573f6902340, scope.project.id=4117990fba5243fcafb57c9a4a5c16d9, user.id=2bb7a19cf45d403ba0f5f43d0a43c152) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:34:59.638207 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7484a04f-38df-4ab4-a13f-e49f604b2726 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:34:59.639787 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7484a04f-38df-4ab4-a13f-e49f604b2726 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:34:59.651682 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 525/1059] 10.222.0.15 () {56 vars in 1374 bytes} [Sun Sep 11 10:34:59 2022] GET /identity/v3/role_assignments?user.id=2bb7a19cf45d403ba0f5f43d0a43c152&scope.project.id=4117990fba5243fcafb57c9a4a5c16d9&role.id=2cb7d73f0ea8414296367573f6902340 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:35:01.034574 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f3eecf-b337-4256-aa49-bc5e9ab197ed None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:01.035119 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f3eecf-b337-4256-aa49-bc5e9ab197ed None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:01.035119 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f3eecf-b337-4256-aa49-bc5e9ab197ed None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:01.037035 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 535/1060] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:35:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:35:01.042246 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8fa1d9-5ccf-4bde-a37d-6fd982e5cd46 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:01.042663 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8fa1d9-5ccf-4bde-a37d-6fd982e5cd46 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:01.042799 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8fa1d9-5ccf-4bde-a37d-6fd982e5cd46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:01.060726 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-2b8fa1d9-5ccf-4bde-a37d-6fd982e5cd46 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:35:01.061903 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-2b8fa1d9-5ccf-4bde-a37d-6fd982e5cd46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:35:01.135756 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 526/1061] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:35:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 10:35:01.276463 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f593ec10-f51c-49f9-9b0a-34dbd37644c2 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:01.276811 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f593ec10-f51c-49f9-9b0a-34dbd37644c2 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:01.277019 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f593ec10-f51c-49f9-9b0a-34dbd37644c2 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:01.279180 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 536/1062] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:35:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:35:01.283819 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342fa637-8adc-4bc5-9372-c64345f994e8 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:01.284103 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342fa637-8adc-4bc5-9372-c64345f994e8 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:01.284373 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342fa637-8adc-4bc5-9372-c64345f994e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:01.304030 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-342fa637-8adc-4bc5-9372-c64345f994e8 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:35:01.305125 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-342fa637-8adc-4bc5-9372-c64345f994e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:35:01.349775 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:01.378227 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 527/1063] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:35:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 10:35:01.403170 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f593ec10-f51c-49f9-9b0a-34dbd37644c2 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:01.406212 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:01.510096 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f593ec10-f51c-49f9-9b0a-34dbd37644c2 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:01.544542 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fac9bb-eed9-4229-b830-6e4212077424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:01.545182 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fac9bb-eed9-4229-b830-6e4212077424 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:01.545461 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fac9bb-eed9-4229-b830-6e4212077424 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:01.545731 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fac9bb-eed9-4229-b830-6e4212077424 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:01.556039 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fac9bb-eed9-4229-b830-6e4212077424 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:35:01.560710 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-11fac9bb-eed9-4229-b830-6e4212077424 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 10:35:01.560710 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:35:01.560710 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:35:01.560710 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:35:01.560710 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:35:01.560710 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:35:01.560710 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:35:01.560710 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:35:01.560710 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:35:01.560710 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:35:01.560710 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:35:01.560710 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:35:01.560710 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:35:01.560710 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:35:01.560710 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:35:01.560710 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:35:01.561641 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:35:01.561641 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:35:01.561641 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:35:01.561641 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:35:01.561641 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:35:01.561641 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:35:01.561641 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:35:01.561641 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:35:01.561641 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:35:01.561641 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:35:01.561641 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:35:01.561641 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:35:01.561641 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:35:01.561641 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:35:01.561641 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:35:01.561641 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:35:01.564438 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:35:01.564438 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:35:01.564438 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:35:01.564438 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:35:01.564438 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:35:01.564438 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:35:01.564438 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:35:01.564438 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:35:01.564438 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:35:01.564438 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:35:01.564438 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 10:35:01.564438 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:35:01.564438 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 537/1064] 10.222.0.15 () {56 vars in 1120 bytes} [Sun Sep 11 10:35:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:35:01.568536 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342fa637-8adc-4bc5-9372-c64345f994e8 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:01.647450 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fac9bb-eed9-4229-b830-6e4212077424 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:01.676687 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342fa637-8adc-4bc5-9372-c64345f994e8 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:01.717620 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:01.723718 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a8db10-6479-4723-be53-a4da10d781d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:01.724531 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a8db10-6479-4723-be53-a4da10d781d5 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:01.724812 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a8db10-6479-4723-be53-a4da10d781d5 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:01.725082 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a8db10-6479-4723-be53-a4da10d781d5 None admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:01.725760 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a8db10-6479-4723-be53-a4da10d781d5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:35:01.726060 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a8db10-6479-4723-be53-a4da10d781d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:01.727766 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a8db10-6479-4723-be53-a4da10d781d5 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:01.736958 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 528/1065] 10.222.0.15 () {56 vars in 1115 bytes} [Sun Sep 11 10:35:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:35:01.742819 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a8db10-6479-4723-be53-a4da10d781d5 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:01.762334 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fac9bb-eed9-4229-b830-6e4212077424 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:01.782586 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:01.801690 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae3f36c-2582-484b-b91d-19d7a1a784c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:01.803237 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae3f36c-2582-484b-b91d-19d7a1a784c6 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:01.803758 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae3f36c-2582-484b-b91d-19d7a1a784c6 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:01.804051 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae3f36c-2582-484b-b91d-19d7a1a784c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:01.852279 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a8db10-6479-4723-be53-a4da10d781d5 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:01.872901 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae3f36c-2582-484b-b91d-19d7a1a784c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:01.874342 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae3f36c-2582-484b-b91d-19d7a1a784c6 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:01.886828 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1979aaa7-58af-48fe-886b-c8c9f4337975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:01.887480 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1979aaa7-58af-48fe-886b-c8c9f4337975 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:01.887767 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1979aaa7-58af-48fe-886b-c8c9f4337975 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:01.888043 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1979aaa7-58af-48fe-886b-c8c9f4337975 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:01.898491 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1979aaa7-58af-48fe-886b-c8c9f4337975 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:01.899961 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1979aaa7-58af-48fe-886b-c8c9f4337975 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:01.913903 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-1979aaa7-58af-48fe-886b-c8c9f4337975 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:35:01.913903 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:35:01.913903 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:35:01.913903 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:35:01.913903 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:35:01.913903 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:35:01.913903 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:35:01.913903 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:35:01.913903 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:35:01.913903 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:35:01.913903 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:35:01.913903 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:35:01.913903 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:35:01.913903 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:35:01.913903 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:35:01.913903 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:35:01.915360 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:35:01.915360 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:35:01.915360 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:35:01.915360 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:35:01.915360 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:35:01.915360 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:35:01.915360 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:35:01.915360 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:35:01.915360 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:35:01.915360 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:35:01.915360 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:35:01.915360 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:35:01.915360 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:35:01.915360 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:35:01.915360 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:35:01.915360 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:35:01.915360 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:35:01.916424 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:35:01.916424 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:35:01.916424 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:35:01.916424 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 529/1066] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:35:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 10:35:01.916424 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481e9a8-5750-4fe4-aa04-cad687143592 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:01.916424 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481e9a8-5750-4fe4-aa04-cad687143592 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:01.916774 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481e9a8-5750-4fe4-aa04-cad687143592 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:01.918029 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 530/1067] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:35:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:35:01.922606 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6481e9a8-5750-4fe4-aa04-cad687143592 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:01.936718 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:01.976096 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 538/1068] 10.222.0.15 () {58 vars in 1282 bytes} [Sun Sep 11 10:35:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 11 10:35:01.977354 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698fe840-58d6-4b05-b2ed-336585ac990d None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:01.977904 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698fe840-58d6-4b05-b2ed-336585ac990d None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:01.978275 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698fe840-58d6-4b05-b2ed-336585ac990d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:01.996495 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-698fe840-58d6-4b05-b2ed-336585ac990d None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:35:01.997683 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-698fe840-58d6-4b05-b2ed-336585ac990d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:35:02.015120 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:02.028645 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6481e9a8-5750-4fe4-aa04-cad687143592 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:02.063219 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df220195-5139-4324-a70e-44b194e776f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:02.063929 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df220195-5139-4324-a70e-44b194e776f0 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:02.064116 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df220195-5139-4324-a70e-44b194e776f0 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:02.064287 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 539/1069] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:35:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 10:35:02.064671 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df220195-5139-4324-a70e-44b194e776f0 None admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:02.065025 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df220195-5139-4324-a70e-44b194e776f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:35:02.065138 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698fe840-58d6-4b05-b2ed-336585ac990d None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:02.065330 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df220195-5139-4324-a70e-44b194e776f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:02.066703 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df220195-5139-4324-a70e-44b194e776f0 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:02.080952 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 531/1070] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:35:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:35:02.084760 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df220195-5139-4324-a70e-44b194e776f0 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:02.167698 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:02.177209 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698fe840-58d6-4b05-b2ed-336585ac990d None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:02.196497 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df220195-5139-4324-a70e-44b194e776f0 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:02.210984 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9af85d5-f5c9-49e1-bb60-c4ac4189a1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:02.211647 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9af85d5-f5c9-49e1-bb60-c4ac4189a1e9 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:02.211931 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9af85d5-f5c9-49e1-bb60-c4ac4189a1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:02.212208 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9af85d5-f5c9-49e1-bb60-c4ac4189a1e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:02.222948 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9af85d5-f5c9-49e1-bb60-c4ac4189a1e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:02.225158 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9af85d5-f5c9-49e1-bb60-c4ac4189a1e9 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:02.231740 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45900796-3e81-464e-931f-cf09a61d260c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:02.232369 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45900796-3e81-464e-931f-cf09a61d260c None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:02.232752 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45900796-3e81-464e-931f-cf09a61d260c None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:02.232983 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45900796-3e81-464e-931f-cf09a61d260c None admin] PATH_INFO: `/v3/users/nova` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:02.240493 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45900796-3e81-464e-931f-cf09a61d260c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:02.241573 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-d9af85d5-f5c9-49e1-bb60-c4ac4189a1e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:35:02.241573 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:35:02.241573 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:35:02.241573 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:35:02.241573 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:35:02.241573 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:35:02.241573 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:35:02.241573 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:35:02.241573 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:35:02.241573 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:35:02.241573 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:35:02.241573 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:35:02.241573 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:35:02.241573 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:35:02.241573 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:35:02.241573 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:35:02.242525 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:35:02.242525 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:35:02.242525 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:35:02.242525 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:35:02.242525 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:35:02.242525 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:35:02.242525 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:35:02.242525 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:35:02.242525 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:35:02.242525 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:35:02.242525 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:35:02.242525 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:35:02.242525 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:35:02.242525 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:35:02.242525 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:35:02.242525 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:35:02.242525 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:35:02.243486 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:35:02.243486 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:35:02.243486 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:35:02.243667 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45900796-3e81-464e-931f-cf09a61d260c None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:02.247752 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 540/1071] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:35:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:35:02.248806 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9af85d5-f5c9-49e1-bb60-c4ac4189a1e9 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:02.254338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-45900796-3e81-464e-931f-cf09a61d260c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 11 10:35:02.254338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:35:02.254338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:35:02.254338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:35:02.254338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:35:02.254338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:35:02.254338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:35:02.254338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:35:02.254338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:35:02.254338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:35:02.254338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:35:02.254338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:35:02.254338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 10:35:02.254338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 10:35:02.254338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 10:35:02.254338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 10:35:02.255881 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:35:02.255881 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:35:02.255881 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 10:35:02.255881 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:35:02.255881 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 10:35:02.255881 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:35:02.255881 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:35:02.255881 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:35:02.255881 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:35:02.255881 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:35:02.255881 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:35:02.255881 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:35:02.255881 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:35:02.255881 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:35:02.255881 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:35:02.255881 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:35:02.255881 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:35:02.255881 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:35:02.256865 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:35:02.256865 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:35:02.256865 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 10:35:02.256865 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 10:35:02.256865 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 10:35:02.256865 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 10:35:02.256865 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 10:35:02.256865 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 10:35:02.256865 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 11 10:35:02.256865 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:35:02.256865 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 532/1072] 10.222.0.15 () {56 vars in 1093 bytes} [Sun Sep 11 10:35:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:35:02.261970 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45900796-3e81-464e-931f-cf09a61d260c None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:02.273741 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:02.358536 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9af85d5-f5c9-49e1-bb60-c4ac4189a1e9 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:02.376059 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45900796-3e81-464e-931f-cf09a61d260c None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:02.392033 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de256aa8-84f0-45b8-9c4f-572b93e22ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:02.392639 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de256aa8-84f0-45b8-9c4f-572b93e22ed4 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:02.392919 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de256aa8-84f0-45b8-9c4f-572b93e22ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:02.393192 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de256aa8-84f0-45b8-9c4f-572b93e22ed4 None admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:02.393826 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de256aa8-84f0-45b8-9c4f-572b93e22ed4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:35:02.394145 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de256aa8-84f0-45b8-9c4f-572b93e22ed4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:02.395524 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de256aa8-84f0-45b8-9c4f-572b93e22ed4 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:02.408457 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 541/1073] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:35:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:35:02.409254 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de256aa8-84f0-45b8-9c4f-572b93e22ed4 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:02.410350 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3178ee0c-4c8f-454b-bb7f-020daf77b294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:02.410970 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3178ee0c-4c8f-454b-bb7f-020daf77b294 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:02.411794 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3178ee0c-4c8f-454b-bb7f-020daf77b294 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:02.412221 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3178ee0c-4c8f-454b-bb7f-020daf77b294 None admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:02.412962 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3178ee0c-4c8f-454b-bb7f-020daf77b294 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:35:02.413268 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3178ee0c-4c8f-454b-bb7f-020daf77b294 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:02.414746 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3178ee0c-4c8f-454b-bb7f-020daf77b294 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:02.422650 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:02.436290 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 533/1074] 10.222.0.15 () {56 vars in 1133 bytes} [Sun Sep 11 10:35:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:35:02.442409 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3178ee0c-4c8f-454b-bb7f-020daf77b294 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:02.516585 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:02.545532 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de256aa8-84f0-45b8-9c4f-572b93e22ed4 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:02.557694 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3178ee0c-4c8f-454b-bb7f-020daf77b294 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:02.585895 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d26389-79d0-4212-81c5-cb1eb7bcb19e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:02.587306 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d26389-79d0-4212-81c5-cb1eb7bcb19e None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:02.587306 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d26389-79d0-4212-81c5-cb1eb7bcb19e None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:02.587306 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d26389-79d0-4212-81c5-cb1eb7bcb19e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:02.598061 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcf759f-fcc6-4b9e-8e82-ff6ae8f1d75e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:02.598691 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcf759f-fcc6-4b9e-8e82-ff6ae8f1d75e None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:02.599702 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcf759f-fcc6-4b9e-8e82-ff6ae8f1d75e None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:02.600173 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcf759f-fcc6-4b9e-8e82-ff6ae8f1d75e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:02.601123 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d26389-79d0-4212-81c5-cb1eb7bcb19e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:35:02.606182 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-f6d26389-79d0-4212-81c5-cb1eb7bcb19e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 10:35:02.606182 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:35:02.606182 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:35:02.606182 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:35:02.606182 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:35:02.606182 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:35:02.606182 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:35:02.606182 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:35:02.606182 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:35:02.606182 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:35:02.606182 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:35:02.606182 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:35:02.606182 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:35:02.606182 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:35:02.606182 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:35:02.606182 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:35:02.607126 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:35:02.607126 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:35:02.607126 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:35:02.607126 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:35:02.607126 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:35:02.607126 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:35:02.607126 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:35:02.607126 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:35:02.607126 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:35:02.607126 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:35:02.607126 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:35:02.607126 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:35:02.607126 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:35:02.607126 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:35:02.607126 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:35:02.607126 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:35:02.608020 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:35:02.608020 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:35:02.608020 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:35:02.608020 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:35:02.608020 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:35:02.608020 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:35:02.608020 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:35:02.608020 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:35:02.608020 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:35:02.608020 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:35:02.608020 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 10:35:02.608020 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:35:02.608020 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 542/1075] 10.222.0.15 () {56 vars in 1096 bytes} [Sun Sep 11 10:35:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:35:02.612110 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcf759f-fcc6-4b9e-8e82-ff6ae8f1d75e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:02.613226 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d26389-79d0-4212-81c5-cb1eb7bcb19e None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:02.613556 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcf759f-fcc6-4b9e-8e82-ff6ae8f1d75e None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:02.630869 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-2fcf759f-fcc6-4b9e-8e82-ff6ae8f1d75e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:35:02.630869 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:35:02.630869 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:35:02.630869 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:35:02.630869 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:35:02.630869 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:35:02.630869 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:35:02.630869 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:35:02.630869 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:35:02.630869 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:35:02.630869 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:35:02.630869 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:35:02.630869 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:35:02.630869 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:35:02.630869 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:35:02.630869 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:35:02.632232 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:35:02.632232 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:35:02.632232 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:35:02.632232 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:35:02.632232 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:35:02.632232 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:35:02.632232 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:35:02.632232 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:35:02.632232 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:35:02.632232 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:35:02.632232 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:35:02.632232 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:35:02.632232 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:35:02.632232 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:35:02.632232 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:35:02.632232 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:35:02.632232 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:35:02.633136 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:35:02.633136 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:35:02.633136 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:35:02.633136 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 534/1076] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:35:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:35:02.640151 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcf759f-fcc6-4b9e-8e82-ff6ae8f1d75e None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:02.686489 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:02.737573 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d26389-79d0-4212-81c5-cb1eb7bcb19e None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:02.760791 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcf759f-fcc6-4b9e-8e82-ff6ae8f1d75e None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:02.773167 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0fd57d-fe72-4765-a0c5-83e7c1049fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:02.773782 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0fd57d-fe72-4765-a0c5-83e7c1049fc9 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:02.774066 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0fd57d-fe72-4765-a0c5-83e7c1049fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:02.774341 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0fd57d-fe72-4765-a0c5-83e7c1049fc9 None admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:02.775001 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0fd57d-fe72-4765-a0c5-83e7c1049fc9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:35:02.775308 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0fd57d-fe72-4765-a0c5-83e7c1049fc9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:02.776710 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0fd57d-fe72-4765-a0c5-83e7c1049fc9 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:02.780099 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:02.785142 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 543/1077] 10.222.0.15 () {56 vars in 1099 bytes} [Sun Sep 11 10:35:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:35:02.791672 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0fd57d-fe72-4765-a0c5-83e7c1049fc9 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:02.798146 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacece19-389a-4aaf-9d2c-39bee4f50396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:02.798759 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacece19-389a-4aaf-9d2c-39bee4f50396 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:02.799043 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacece19-389a-4aaf-9d2c-39bee4f50396 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:02.799318 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacece19-389a-4aaf-9d2c-39bee4f50396 None admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:02.799991 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacece19-389a-4aaf-9d2c-39bee4f50396 None admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:35:02.800302 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacece19-389a-4aaf-9d2c-39bee4f50396 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:02.801657 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacece19-389a-4aaf-9d2c-39bee4f50396 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:02.817976 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 535/1078] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:35:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:35:02.818980 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacece19-389a-4aaf-9d2c-39bee4f50396 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:02.891748 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0fd57d-fe72-4765-a0c5-83e7c1049fc9 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:02.917102 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:02.918355 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacece19-389a-4aaf-9d2c-39bee4f50396 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:02.926023 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266a73ff-6e7f-40b3-b7b0-6b4cddc53492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:02.926648 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266a73ff-6e7f-40b3-b7b0-6b4cddc53492 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:02.927058 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266a73ff-6e7f-40b3-b7b0-6b4cddc53492 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:02.927336 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266a73ff-6e7f-40b3-b7b0-6b4cddc53492 None admin] PATH_INFO: `/v3/users/2bb7a19cf45d403ba0f5f43d0a43c152` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:02.928490 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266a73ff-6e7f-40b3-b7b0-6b4cddc53492 None admin] RBAC: Authorizing `identity:get_user(user_id=2bb7a19cf45d403ba0f5f43d0a43c152)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:02.929915 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266a73ff-6e7f-40b3-b7b0-6b4cddc53492 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:02.931672 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 544/1079] 10.222.0.15 () {56 vars in 1177 bytes} [Sun Sep 11 10:35:02 2022] GET /identity/v3/users/2bb7a19cf45d403ba0f5f43d0a43c152 => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:35:02.937195 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266a73ff-6e7f-40b3-b7b0-6b4cddc53492 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:02.953129 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c88337-cafc-4ca4-85f0-c794d0c15088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:02.953759 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c88337-cafc-4ca4-85f0-c794d0c15088 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:02.954039 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c88337-cafc-4ca4-85f0-c794d0c15088 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:02.954313 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c88337-cafc-4ca4-85f0-c794d0c15088 None admin] PATH_INFO: `/v3/projects/service` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:02.961470 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c88337-cafc-4ca4-85f0-c794d0c15088 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:02.962905 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c88337-cafc-4ca4-85f0-c794d0c15088 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:02.973883 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-57c88337-cafc-4ca4-85f0-c794d0c15088 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:35:02.973883 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:35:02.973883 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:35:02.973883 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:35:02.973883 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:35:02.973883 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:35:02.973883 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:35:02.973883 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:35:02.973883 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:35:02.973883 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:35:02.973883 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:35:02.973883 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:35:02.973883 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 10:35:02.973883 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 10:35:02.973883 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 10:35:02.975013 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 10:35:02.975013 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:35:02.975013 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:35:02.975013 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:35:02.975013 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:35:02.975013 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:35:02.975013 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:35:02.975013 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:35:02.975013 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:35:02.975013 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:35:02.975013 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:35:02.975013 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:35:02.975013 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:35:02.975013 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:35:02.975013 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:35:02.975013 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:35:02.975013 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:35:02.975966 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 10:35:02.975966 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 10:35:02.975966 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 10:35:02.975966 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 10:35:02.975966 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 10:35:02.975966 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 10:35:02.975966 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:35:02.975966 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:35:02.975966 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 536/1080] 10.222.0.15 () {56 vars in 1111 bytes} [Sun Sep 11 10:35:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:35:02.980918 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c88337-cafc-4ca4-85f0-c794d0c15088 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:03.020286 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:03.036269 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266a73ff-6e7f-40b3-b7b0-6b4cddc53492 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:03.069346 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bc4c71-e8cc-4677-a7ee-f3f9e5b3b00d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:03.069989 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bc4c71-e8cc-4677-a7ee-f3f9e5b3b00d None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:03.070243 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bc4c71-e8cc-4677-a7ee-f3f9e5b3b00d None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:03.070518 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bc4c71-e8cc-4677-a7ee-f3f9e5b3b00d None admin] PATH_INFO: `/v3/projects/4117990fba5243fcafb57c9a4a5c16d9` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:03.071661 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bc4c71-e8cc-4677-a7ee-f3f9e5b3b00d None admin] RBAC: Authorizing `identity:get_project(project_id=4117990fba5243fcafb57c9a4a5c16d9)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:03.073120 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bc4c71-e8cc-4677-a7ee-f3f9e5b3b00d None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:03.074757 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 545/1081] 10.222.0.15 () {56 vars in 1186 bytes} [Sun Sep 11 10:35:02 2022] GET /identity/v3/projects/4117990fba5243fcafb57c9a4a5c16d9 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:35:03.079944 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c88337-cafc-4ca4-85f0-c794d0c15088 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:03.080451 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bc4c71-e8cc-4677-a7ee-f3f9e5b3b00d None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:03.114083 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7867065-b159-4210-ad39-b4acf1639663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:03.115566 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7867065-b159-4210-ad39-b4acf1639663 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:03.115755 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7867065-b159-4210-ad39-b4acf1639663 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:03.116035 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7867065-b159-4210-ad39-b4acf1639663 None admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:03.116720 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7867065-b159-4210-ad39-b4acf1639663 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:35:03.117027 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7867065-b159-4210-ad39-b4acf1639663 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:03.118444 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7867065-b159-4210-ad39-b4acf1639663 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:03.130290 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 537/1082] 10.222.0.15 () {56 vars in 1148 bytes} [Sun Sep 11 10:35:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:35:03.136339 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7867065-b159-4210-ad39-b4acf1639663 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:03.141521 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:03.181572 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bc4c71-e8cc-4677-a7ee-f3f9e5b3b00d None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:03.217324 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fdd895-80ac-4461-b221-a53509062eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:03.217903 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fdd895-80ac-4461-b221-a53509062eb5 None admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:03.218182 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fdd895-80ac-4461-b221-a53509062eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:03.218455 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fdd895-80ac-4461-b221-a53509062eb5 None admin] PATH_INFO: `/v3/projects/4117990fba5243fcafb57c9a4a5c16d9/users/2bb7a19cf45d403ba0f5f43d0a43c152/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:03.220373 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fdd895-80ac-4461-b221-a53509062eb5 None admin] RBAC: Authorizing `identity:create_grant(project_id=4117990fba5243fcafb57c9a4a5c16d9, user_id=2bb7a19cf45d403ba0f5f43d0a43c152, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:03.221792 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fdd895-80ac-4461-b221-a53509062eb5 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:03.232298 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:03.235911 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 546/1083] 10.222.0.15 () {58 vars in 1439 bytes} [Sun Sep 11 10:35:03 2022] PUT /identity/v3/projects/4117990fba5243fcafb57c9a4a5c16d9/users/2bb7a19cf45d403ba0f5f43d0a43c152/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:35:03.236497 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7867065-b159-4210-ad39-b4acf1639663 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:03.279183 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bea792-64b4-4648-bdc6-d02a4406f844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:03.279900 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bea792-64b4-4648-bdc6-d02a4406f844 None admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:03.280176 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bea792-64b4-4648-bdc6-d02a4406f844 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:03.280444 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bea792-64b4-4648-bdc6-d02a4406f844 None admin] PATH_INFO: `/v3/projects/5e1f845f99f04202b83621cb65e0adc2/users/2bd5356829b6454993bab9bad2f79fed/roles/55b167894e584c9b8303c60cf3a75a65` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:03.282660 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bea792-64b4-4648-bdc6-d02a4406f844 None admin] RBAC: Authorizing `identity:create_grant(project_id=5e1f845f99f04202b83621cb65e0adc2, user_id=2bd5356829b6454993bab9bad2f79fed, role_id=55b167894e584c9b8303c60cf3a75a65)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:03.284140 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bea792-64b4-4648-bdc6-d02a4406f844 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:03.299215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 538/1084] 10.222.0.15 () {58 vars in 1439 bytes} [Sun Sep 11 10:35:03 2022] PUT /identity/v3/projects/5e1f845f99f04202b83621cb65e0adc2/users/2bd5356829b6454993bab9bad2f79fed/roles/55b167894e584c9b8303c60cf3a75a65 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 10:35:05.406242 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd348d26-5c39-48c0-a337-6d7522b84456 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:05.406784 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd348d26-5c39-48c0-a337-6d7522b84456 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:05.406784 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd348d26-5c39-48c0-a337-6d7522b84456 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:05.408162 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 547/1085] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:35:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:35:05.414128 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7245a426-3627-47c2-ba0c-1551a02ca963 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:05.414567 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7245a426-3627-47c2-ba0c-1551a02ca963 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:05.414681 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7245a426-3627-47c2-ba0c-1551a02ca963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:05.433206 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-7245a426-3627-47c2-ba0c-1551a02ca963 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:35:05.434392 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-7245a426-3627-47c2-ba0c-1551a02ca963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:35:05.462638 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d027cff4-05c1-46a1-ab80-42f37ac4a0ef None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:05.462925 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d027cff4-05c1-46a1-ab80-42f37ac4a0ef None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:05.463201 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d027cff4-05c1-46a1-ab80-42f37ac4a0ef None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:05.464582 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 548/1086] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:35:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 10:35:05.469432 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f34c3e-398b-474e-925b-ccedb9c00c70 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:05.469717 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f34c3e-398b-474e-925b-ccedb9c00c70 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:05.469993 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f34c3e-398b-474e-925b-ccedb9c00c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:05.488462 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-b3f34c3e-398b-474e-925b-ccedb9c00c70 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:35:05.490254 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-b3f34c3e-398b-474e-925b-ccedb9c00c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:35:05.501881 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 539/1087] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:35:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 10:35:05.555166 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 549/1088] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:35:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 10:35:05.780749 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7245a426-3627-47c2-ba0c-1551a02ca963 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:05.825466 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f34c3e-398b-474e-925b-ccedb9c00c70 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:05.869065 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:05.879777 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:05.882352 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7245a426-3627-47c2-ba0c-1551a02ca963 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:05.918020 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d370ac71-8f51-47f2-a292-d2332dfe9239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:05.918645 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d370ac71-8f51-47f2-a292-d2332dfe9239 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:05.918929 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d370ac71-8f51-47f2-a292-d2332dfe9239 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:05.919209 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d370ac71-8f51-47f2-a292-d2332dfe9239 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:05.927916 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f34c3e-398b-474e-925b-ccedb9c00c70 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:05.929325 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d370ac71-8f51-47f2-a292-d2332dfe9239 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:35:05.934061 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-d370ac71-8f51-47f2-a292-d2332dfe9239 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 10:35:05.934061 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:35:05.934061 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:35:05.934061 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:35:05.934061 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:35:05.934061 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:35:05.934061 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:35:05.934061 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:35:05.934061 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:35:05.934061 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:35:05.934061 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:35:05.934061 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:35:05.934061 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:35:05.934061 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:35:05.934061 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:35:05.934061 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:35:05.935122 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:35:05.935122 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:35:05.935122 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:35:05.935122 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:35:05.935122 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:35:05.935122 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:35:05.935122 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:35:05.935122 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:35:05.935122 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:35:05.935122 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:35:05.935122 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:35:05.935122 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:35:05.935122 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:35:05.935122 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:35:05.935122 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:35:05.935122 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:35:05.936013 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:35:05.936013 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:35:05.936013 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:35:05.936013 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:35:05.936013 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:35:05.936013 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:35:05.936013 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:35:05.936013 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:35:05.936013 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:35:05.936013 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:35:05.936013 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 10:35:05.936013 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:35:05.936013 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 540/1089] 10.222.0.15 () {56 vars in 1096 bytes} [Sun Sep 11 10:35:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:35:05.941106 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d370ac71-8f51-47f2-a292-d2332dfe9239 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:05.962410 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5c8cd1-5521-46cc-a55d-44a39a580174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:05.963034 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c8cd1-5521-46cc-a55d-44a39a580174 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:05.963310 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c8cd1-5521-46cc-a55d-44a39a580174 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:05.963613 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c8cd1-5521-46cc-a55d-44a39a580174 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:05.973141 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5c8cd1-5521-46cc-a55d-44a39a580174 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:35:05.979025 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-fc5c8cd1-5521-46cc-a55d-44a39a580174 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 10:35:05.979025 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:35:05.979025 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:35:05.979025 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:35:05.979025 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:35:05.979025 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:35:05.979025 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:35:05.979025 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:35:05.979025 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:35:05.979025 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:35:05.979025 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:35:05.979025 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:35:05.979025 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:35:05.979025 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:35:05.979025 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:35:05.979025 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:35:05.980021 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:35:05.980021 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:35:05.980021 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:35:05.980021 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:35:05.980021 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:35:05.980021 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:35:05.980021 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:35:05.980021 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:35:05.980021 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:35:05.980021 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:35:05.980021 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:35:05.980021 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:35:05.980021 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:35:05.980021 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:35:05.980021 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:35:05.980021 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:35:05.980893 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:35:05.980893 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:35:05.980893 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:35:05.980893 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:35:05.980893 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:35:05.980893 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:35:05.980893 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:35:05.980893 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:35:05.980893 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:35:05.980893 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:35:05.980893 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 10:35:05.980893 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:35:05.980893 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 550/1090] 10.222.0.15 () {56 vars in 1120 bytes} [Sun Sep 11 10:35:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 10:35:05.986078 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5c8cd1-5521-46cc-a55d-44a39a580174 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:06.043311 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d370ac71-8f51-47f2-a292-d2332dfe9239 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:06.077122 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9c23b5-94c8-4c0a-b5ac-72e54062eb83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:06.077724 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c23b5-94c8-4c0a-b5ac-72e54062eb83 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:06.078022 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c23b5-94c8-4c0a-b5ac-72e54062eb83 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:06.078288 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c23b5-94c8-4c0a-b5ac-72e54062eb83 None admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:06.078943 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9c23b5-94c8-4c0a-b5ac-72e54062eb83 None admin] RBAC: Adding query filter params (name=admin) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:35:06.079248 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9c23b5-94c8-4c0a-b5ac-72e54062eb83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:06.080644 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9c23b5-94c8-4c0a-b5ac-72e54062eb83 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:06.084245 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5c8cd1-5521-46cc-a55d-44a39a580174 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:06.088668 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 541/1091] 10.222.0.15 () {56 vars in 1099 bytes} [Sun Sep 11 10:35:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:35:06.094621 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9c23b5-94c8-4c0a-b5ac-72e54062eb83 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:06.097234 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:06.108294 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:06.120442 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11eb442-c9bc-4a1f-ba7e-aeb406e29e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:06.121044 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11eb442-c9bc-4a1f-ba7e-aeb406e29e0c None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:06.121319 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11eb442-c9bc-4a1f-ba7e-aeb406e29e0c None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:06.121595 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11eb442-c9bc-4a1f-ba7e-aeb406e29e0c None admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:06.122244 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11eb442-c9bc-4a1f-ba7e-aeb406e29e0c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:35:06.122540 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11eb442-c9bc-4a1f-ba7e-aeb406e29e0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:06.123939 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11eb442-c9bc-4a1f-ba7e-aeb406e29e0c None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:06.132409 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 551/1092] 10.222.0.15 () {56 vars in 1115 bytes} [Sun Sep 11 10:35:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:35:06.138106 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11eb442-c9bc-4a1f-ba7e-aeb406e29e0c None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:06.197270 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9c23b5-94c8-4c0a-b5ac-72e54062eb83 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:06.230583 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c240283f-0eb4-41bf-912c-0d8e6a4d308f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:06.231210 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240283f-0eb4-41bf-912c-0d8e6a4d308f None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:06.231513 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240283f-0eb4-41bf-912c-0d8e6a4d308f None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:06.231789 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240283f-0eb4-41bf-912c-0d8e6a4d308f None admin] PATH_INFO: `/v3/users/2bb7a19cf45d403ba0f5f43d0a43c152` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:06.232905 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c240283f-0eb4-41bf-912c-0d8e6a4d308f None admin] RBAC: Authorizing `identity:get_user(user_id=2bb7a19cf45d403ba0f5f43d0a43c152)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:06.234326 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c240283f-0eb4-41bf-912c-0d8e6a4d308f None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:06.235880 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11eb442-c9bc-4a1f-ba7e-aeb406e29e0c None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:06.236008 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 542/1093] 10.222.0.15 () {56 vars in 1177 bytes} [Sun Sep 11 10:35:06 2022] GET /identity/v3/users/2bb7a19cf45d403ba0f5f43d0a43c152 => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:35:06.242313 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c240283f-0eb4-41bf-912c-0d8e6a4d308f None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:06.272169 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c832cb4b-5f44-4d4a-8729-4c9385afac66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:06.272796 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c832cb4b-5f44-4d4a-8729-4c9385afac66 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:06.273072 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c832cb4b-5f44-4d4a-8729-4c9385afac66 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:06.273343 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c832cb4b-5f44-4d4a-8729-4c9385afac66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:06.283967 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c832cb4b-5f44-4d4a-8729-4c9385afac66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:06.285424 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c832cb4b-5f44-4d4a-8729-4c9385afac66 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:06.301111 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-c832cb4b-5f44-4d4a-8729-4c9385afac66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:35:06.301111 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:35:06.301111 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:35:06.301111 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:35:06.301111 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:35:06.301111 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:35:06.301111 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:35:06.301111 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:35:06.301111 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:35:06.301111 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:35:06.301111 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:35:06.301111 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:35:06.301111 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:35:06.301111 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:35:06.301111 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:35:06.301111 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:35:06.302321 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:35:06.302321 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:35:06.302321 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:35:06.302321 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:35:06.302321 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:35:06.302321 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:35:06.302321 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:35:06.302321 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:35:06.302321 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:35:06.302321 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:35:06.302321 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:35:06.302321 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:35:06.302321 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:35:06.302321 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:35:06.302321 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:35:06.302321 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:35:06.302321 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:35:06.303563 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:35:06.303563 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:35:06.303563 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:35:06.303563 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 552/1094] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:35:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:35:06.308924 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c832cb4b-5f44-4d4a-8729-4c9385afac66 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:06.321273 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:06.336403 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:06.343926 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c240283f-0eb4-41bf-912c-0d8e6a4d308f None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:06.378226 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297313e0-b078-4168-ade9-b8f78e7c5df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:06.378840 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297313e0-b078-4168-ade9-b8f78e7c5df7 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:06.379122 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297313e0-b078-4168-ade9-b8f78e7c5df7 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:06.379390 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297313e0-b078-4168-ade9-b8f78e7c5df7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:06.409546 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c832cb4b-5f44-4d4a-8729-4c9385afac66 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:06.442049 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d048dcab-a21e-4ed2-bb30-739ced2cdbb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:06.442760 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d048dcab-a21e-4ed2-bb30-739ced2cdbb0 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:06.443037 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d048dcab-a21e-4ed2-bb30-739ced2cdbb0 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:06.443299 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d048dcab-a21e-4ed2-bb30-739ced2cdbb0 None admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:06.443957 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d048dcab-a21e-4ed2-bb30-739ced2cdbb0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:35:06.444255 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d048dcab-a21e-4ed2-bb30-739ced2cdbb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:06.445606 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d048dcab-a21e-4ed2-bb30-739ced2cdbb0 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:06.449125 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297313e0-b078-4168-ade9-b8f78e7c5df7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:06.450555 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297313e0-b078-4168-ade9-b8f78e7c5df7 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:06.456896 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 553/1095] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:35:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:35:06.463166 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d048dcab-a21e-4ed2-bb30-739ced2cdbb0 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:06.533076 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:06.545008 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:06.556216 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 543/1096] 10.222.0.15 () {58 vars in 1282 bytes} [Sun Sep 11 10:35:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 11 10:35:06.560509 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d048dcab-a21e-4ed2-bb30-739ced2cdbb0 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:06.561945 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297313e0-b078-4168-ade9-b8f78e7c5df7 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:06.594817 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d5e82d-573d-46c9-83c2-0a3a853621df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:06.595464 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d5e82d-573d-46c9-83c2-0a3a853621df None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:06.595750 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d5e82d-573d-46c9-83c2-0a3a853621df None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:06.596027 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d5e82d-573d-46c9-83c2-0a3a853621df None admin] PATH_INFO: `/v3/users/nova` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:06.603380 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d5e82d-573d-46c9-83c2-0a3a853621df None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:06.604951 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d5e82d-573d-46c9-83c2-0a3a853621df None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:06.616707 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-55d5e82d-573d-46c9-83c2-0a3a853621df None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 11 10:35:06.616707 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:35:06.616707 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:35:06.616707 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:35:06.616707 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:35:06.616707 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:35:06.616707 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:35:06.616707 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:35:06.616707 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:35:06.616707 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:35:06.616707 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:35:06.616707 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:35:06.616707 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 10:35:06.616707 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 10:35:06.616707 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 10:35:06.616707 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 10:35:06.617731 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:35:06.617731 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:35:06.617731 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 10:35:06.617731 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:35:06.617731 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 10:35:06.617731 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:35:06.617731 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:35:06.617731 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:35:06.617731 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:35:06.617731 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:35:06.617731 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:35:06.617731 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:35:06.617731 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:35:06.617731 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:35:06.617731 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:35:06.617731 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:35:06.617731 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:35:06.617731 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:35:06.618690 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:35:06.618690 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:35:06.618690 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 10:35:06.618690 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 10:35:06.618690 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 10:35:06.618690 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 10:35:06.618690 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 10:35:06.618690 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 10:35:06.618690 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 11 10:35:06.618690 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:35:06.618690 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 554/1097] 10.222.0.15 () {56 vars in 1093 bytes} [Sun Sep 11 10:35:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 10:35:06.623522 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d5e82d-573d-46c9-83c2-0a3a853621df None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:06.663024 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297313e0-b078-4168-ade9-b8f78e7c5df7 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:06.696959 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cef45f-1636-42d6-b119-344edd7adcee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:06.697591 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cef45f-1636-42d6-b119-344edd7adcee None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:06.697874 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cef45f-1636-42d6-b119-344edd7adcee None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:06.698147 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cef45f-1636-42d6-b119-344edd7adcee None admin] PATH_INFO: `/v3/projects/4117990fba5243fcafb57c9a4a5c16d9` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:06.699258 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cef45f-1636-42d6-b119-344edd7adcee None admin] RBAC: Authorizing `identity:get_project(project_id=4117990fba5243fcafb57c9a4a5c16d9)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:06.700736 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cef45f-1636-42d6-b119-344edd7adcee None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:06.702354 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 544/1098] 10.222.0.15 () {56 vars in 1186 bytes} [Sun Sep 11 10:35:06 2022] GET /identity/v3/projects/4117990fba5243fcafb57c9a4a5c16d9 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:35:06.708917 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cef45f-1636-42d6-b119-344edd7adcee None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:06.722286 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d5e82d-573d-46c9-83c2-0a3a853621df None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:06.755938 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dc9832-9371-4cc1-9c8c-13ba92ba8de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:06.756133 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:06.756537 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc9832-9371-4cc1-9c8c-13ba92ba8de8 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:06.756816 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc9832-9371-4cc1-9c8c-13ba92ba8de8 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:06.757135 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc9832-9371-4cc1-9c8c-13ba92ba8de8 None admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:06.757808 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dc9832-9371-4cc1-9c8c-13ba92ba8de8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:35:06.758110 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dc9832-9371-4cc1-9c8c-13ba92ba8de8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:06.759571 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dc9832-9371-4cc1-9c8c-13ba92ba8de8 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:06.776738 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 555/1099] 10.222.0.15 () {56 vars in 1133 bytes} [Sun Sep 11 10:35:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:35:06.782359 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dc9832-9371-4cc1-9c8c-13ba92ba8de8 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:06.785635 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:06.816186 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cef45f-1636-42d6-b119-344edd7adcee None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:06.851505 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc77334-0dac-434f-8484-3ce59c2e28ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:06.853118 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc77334-0dac-434f-8484-3ce59c2e28ca None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:06.853404 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc77334-0dac-434f-8484-3ce59c2e28ca None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:06.853682 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc77334-0dac-434f-8484-3ce59c2e28ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:06.854402 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc77334-0dac-434f-8484-3ce59c2e28ca None admin] RBAC: Adding query filter params (role.id=2cb7d73f0ea8414296367573f6902340, scope.project.id=4117990fba5243fcafb57c9a4a5c16d9, user.id=2bb7a19cf45d403ba0f5f43d0a43c152) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:35:06.854713 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc77334-0dac-434f-8484-3ce59c2e28ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:06.856184 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc77334-0dac-434f-8484-3ce59c2e28ca None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:06.862952 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 545/1100] 10.222.0.15 () {56 vars in 1374 bytes} [Sun Sep 11 10:35:06 2022] GET /identity/v3/role_assignments?user.id=2bb7a19cf45d403ba0f5f43d0a43c152&scope.project.id=4117990fba5243fcafb57c9a4a5c16d9&role.id=2cb7d73f0ea8414296367573f6902340 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:35:06.882793 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dc9832-9371-4cc1-9c8c-13ba92ba8de8 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:06.919394 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e016318a-f69a-4b99-a677-d1cf9747f384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:06.920050 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e016318a-f69a-4b99-a677-d1cf9747f384 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:06.920328 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e016318a-f69a-4b99-a677-d1cf9747f384 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:06.920598 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e016318a-f69a-4b99-a677-d1cf9747f384 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:06.991834 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e016318a-f69a-4b99-a677-d1cf9747f384 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:06.993303 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e016318a-f69a-4b99-a677-d1cf9747f384 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:06.996243 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:07.095697 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 556/1101] 10.222.0.15 () {58 vars in 1282 bytes} [Sun Sep 11 10:35:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 11 10:35:07.101586 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc77334-0dac-434f-8484-3ce59c2e28ca None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:07.203447 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc77334-0dac-434f-8484-3ce59c2e28ca None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:07.213612 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:07.237909 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fa5fa0-73aa-467f-8eb6-22ec5635bb9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:07.238523 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa5fa0-73aa-467f-8eb6-22ec5635bb9c None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:07.238801 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa5fa0-73aa-467f-8eb6-22ec5635bb9c None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:07.239071 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa5fa0-73aa-467f-8eb6-22ec5635bb9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:07.250152 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fa5fa0-73aa-467f-8eb6-22ec5635bb9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:07.251624 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fa5fa0-73aa-467f-8eb6-22ec5635bb9c None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:07.265999 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-22fa5fa0-73aa-467f-8eb6-22ec5635bb9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:35:07.265999 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:35:07.265999 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:35:07.265999 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:35:07.265999 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:35:07.265999 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:35:07.265999 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:35:07.265999 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:35:07.265999 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:35:07.265999 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:35:07.265999 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:35:07.265999 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:35:07.265999 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:35:07.265999 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:35:07.265999 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:35:07.265999 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:35:07.266964 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:35:07.266964 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:35:07.266964 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:35:07.266964 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:35:07.266964 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:35:07.266964 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:35:07.266964 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:35:07.266964 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:35:07.266964 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:35:07.266964 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:35:07.266964 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:35:07.266964 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:35:07.266964 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:35:07.266964 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:35:07.266964 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:35:07.266964 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:35:07.266964 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:35:07.267922 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:35:07.267922 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:35:07.267922 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:35:07.267922 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 546/1102] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:35:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:35:07.272779 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e016318a-f69a-4b99-a677-d1cf9747f384 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:07.371323 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e016318a-f69a-4b99-a677-d1cf9747f384 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:07.373569 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:07.405301 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e39c557-4a95-450b-a0cb-23bd66098e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:07.405906 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e39c557-4a95-450b-a0cb-23bd66098e54 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:07.406181 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e39c557-4a95-450b-a0cb-23bd66098e54 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:07.406447 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e39c557-4a95-450b-a0cb-23bd66098e54 None admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:07.407072 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e39c557-4a95-450b-a0cb-23bd66098e54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:35:07.407369 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e39c557-4a95-450b-a0cb-23bd66098e54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:07.408746 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e39c557-4a95-450b-a0cb-23bd66098e54 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:07.420393 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 557/1103] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:35:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:35:07.426044 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fa5fa0-73aa-467f-8eb6-22ec5635bb9c None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:07.524234 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fa5fa0-73aa-467f-8eb6-22ec5635bb9c None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:07.557759 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8687f3-a868-451b-a745-325d726c8e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:07.558379 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8687f3-a868-451b-a745-325d726c8e88 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:07.558656 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8687f3-a868-451b-a745-325d726c8e88 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:07.558922 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8687f3-a868-451b-a745-325d726c8e88 None admin] PATH_INFO: `/v3/projects/service` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:07.565434 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8687f3-a868-451b-a745-325d726c8e88 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:07.566850 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8687f3-a868-451b-a745-325d726c8e88 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:07.578541 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-9f8687f3-a868-451b-a745-325d726c8e88 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:35:07.578541 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:35:07.578541 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:35:07.578541 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:35:07.578541 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:35:07.578541 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:35:07.578541 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:35:07.578541 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:35:07.578541 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:35:07.578541 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:35:07.578541 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:35:07.578541 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:35:07.578541 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 10:35:07.578541 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 10:35:07.578541 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 10:35:07.579653 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 10:35:07.579653 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:35:07.579653 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:35:07.579653 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:35:07.579653 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:35:07.579653 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:35:07.579653 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:35:07.579653 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:35:07.579653 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:35:07.579653 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:35:07.579653 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:35:07.579653 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:35:07.579653 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:35:07.579653 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:35:07.579653 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:35:07.579653 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:35:07.579653 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:35:07.580587 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 10:35:07.580587 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 10:35:07.580587 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 10:35:07.580587 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 10:35:07.580587 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 10:35:07.580587 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 10:35:07.580587 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:35:07.580587 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:35:07.580587 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 547/1104] 10.222.0.15 () {56 vars in 1111 bytes} [Sun Sep 11 10:35:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 10:35:07.586183 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e39c557-4a95-450b-a0cb-23bd66098e54 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:07.684778 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e39c557-4a95-450b-a0cb-23bd66098e54 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:07.717261 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94aa260-e031-4218-8bbd-dd4983754216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:07.717866 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94aa260-e031-4218-8bbd-dd4983754216 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:07.718142 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94aa260-e031-4218-8bbd-dd4983754216 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:07.718413 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94aa260-e031-4218-8bbd-dd4983754216 None admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:07.719102 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94aa260-e031-4218-8bbd-dd4983754216 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:35:07.719402 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94aa260-e031-4218-8bbd-dd4983754216 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:07.720840 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94aa260-e031-4218-8bbd-dd4983754216 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:07.733474 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 558/1105] 10.222.0.15 () {56 vars in 1148 bytes} [Sun Sep 11 10:35:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:35:07.740770 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8687f3-a868-451b-a745-325d726c8e88 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:07.766992 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:07.844380 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8687f3-a868-451b-a745-325d726c8e88 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:07.877838 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459077cd-02bc-4558-908c-d46c297d67d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:07.878431 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459077cd-02bc-4558-908c-d46c297d67d1 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:07.878708 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459077cd-02bc-4558-908c-d46c297d67d1 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:07.878977 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459077cd-02bc-4558-908c-d46c297d67d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:07.879725 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459077cd-02bc-4558-908c-d46c297d67d1 None admin] RBAC: Adding query filter params (role.id=55b167894e584c9b8303c60cf3a75a65, scope.project.id=5e1f845f99f04202b83621cb65e0adc2, user.id=2bd5356829b6454993bab9bad2f79fed) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:35:07.880615 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459077cd-02bc-4558-908c-d46c297d67d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:07.882027 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459077cd-02bc-4558-908c-d46c297d67d1 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:07.887783 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 548/1106] 10.222.0.15 () {56 vars in 1374 bytes} [Sun Sep 11 10:35:07 2022] GET /identity/v3/role_assignments?user.id=2bd5356829b6454993bab9bad2f79fed&scope.project.id=5e1f845f99f04202b83621cb65e0adc2&role.id=55b167894e584c9b8303c60cf3a75a65 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:35:09.048357 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09953b73-808c-4e9f-bf42-474360b3c0ec None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:09.048895 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09953b73-808c-4e9f-bf42-474360b3c0ec None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:09.049065 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09953b73-808c-4e9f-bf42-474360b3c0ec None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:09.050634 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 559/1107] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:35:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:35:09.056715 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0510ce54-f64f-4bc7-81bc-8756281a650b None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:09.057111 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0510ce54-f64f-4bc7-81bc-8756281a650b None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:09.057281 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0510ce54-f64f-4bc7-81bc-8756281a650b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:09.075244 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-0510ce54-f64f-4bc7-81bc-8756281a650b None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:35:09.076457 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-0510ce54-f64f-4bc7-81bc-8756281a650b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:35:09.140243 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 549/1108] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:35:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 10:35:09.410236 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09953b73-808c-4e9f-bf42-474360b3c0ec None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:09.435157 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:09.509296 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09953b73-808c-4e9f-bf42-474360b3c0ec None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:09.542844 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9146ca34-ee2b-4bc3-be2c-8ac6c4038b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:09.543488 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9146ca34-ee2b-4bc3-be2c-8ac6c4038b74 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:09.543772 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9146ca34-ee2b-4bc3-be2c-8ac6c4038b74 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:09.544043 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9146ca34-ee2b-4bc3-be2c-8ac6c4038b74 None admin] PATH_INFO: `/v3/domains/default` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:09.545135 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9146ca34-ee2b-4bc3-be2c-8ac6c4038b74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:09.546565 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9146ca34-ee2b-4bc3-be2c-8ac6c4038b74 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:09.548142 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 560/1109] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:35:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:35:09.553974 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0510ce54-f64f-4bc7-81bc-8756281a650b None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:09.570862 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:09.651341 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0510ce54-f64f-4bc7-81bc-8756281a650b None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:09.684665 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714a7d80-51a0-46f7-b650-118e17d71ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:09.685264 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a7d80-51a0-46f7-b650-118e17d71ce1 None admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:09.685544 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a7d80-51a0-46f7-b650-118e17d71ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:09.685816 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a7d80-51a0-46f7-b650-118e17d71ce1 None admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:09.686465 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714a7d80-51a0-46f7-b650-118e17d71ce1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:09.687905 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714a7d80-51a0-46f7-b650-118e17d71ce1 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:09.709218 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 550/1110] 10.222.0.15 () {60 vars in 1132 bytes} [Sun Sep 11 10:35:09 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:35:11.912031 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02491af3-4399-420b-a35c-80a221a599d9 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:11.912565 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02491af3-4399-420b-a35c-80a221a599d9 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:11.912565 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02491af3-4399-420b-a35c-80a221a599d9 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:11.913898 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 561/1111] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:35:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:35:11.919151 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a31b1-ef56-4aef-9382-a3dca1640730 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:11.919568 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a31b1-ef56-4aef-9382-a3dca1640730 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:11.919767 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a31b1-ef56-4aef-9382-a3dca1640730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:11.938013 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-b73a31b1-ef56-4aef-9382-a3dca1640730 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:35:11.939183 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-b73a31b1-ef56-4aef-9382-a3dca1640730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:35:12.003974 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 551/1112] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:35:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 10:35:12.269219 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02491af3-4399-420b-a35c-80a221a599d9 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:12.358769 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:12.369144 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02491af3-4399-420b-a35c-80a221a599d9 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:12.402204 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cde19b2-031c-454b-acd4-ef484f1671c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:12.402825 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cde19b2-031c-454b-acd4-ef484f1671c2 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:12.403103 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cde19b2-031c-454b-acd4-ef484f1671c2 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:12.403374 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cde19b2-031c-454b-acd4-ef484f1671c2 None admin] PATH_INFO: `/v3/roles/cdb3aa1bd85b4694964629ff7cf8039b` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:12.404758 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cde19b2-031c-454b-acd4-ef484f1671c2 None admin] RBAC: Authorizing `identity:get_role(role_id=cdb3aa1bd85b4694964629ff7cf8039b)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:12.406117 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cde19b2-031c-454b-acd4-ef484f1671c2 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:12.407437 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 562/1113] 10.222.0.15 () {56 vars in 1177 bytes} [Sun Sep 11 10:35:12 2022] GET /identity/v3/roles/cdb3aa1bd85b4694964629ff7cf8039b => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:35:12.413434 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73a31b1-ef56-4aef-9382-a3dca1640730 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:12.415590 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:12.511689 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73a31b1-ef56-4aef-9382-a3dca1640730 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:12.544480 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d499ca59-ed88-4e47-a4f8-d3fc8738954d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:12.545036 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d499ca59-ed88-4e47-a4f8-d3fc8738954d None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:12.545314 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d499ca59-ed88-4e47-a4f8-d3fc8738954d None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:12.545583 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d499ca59-ed88-4e47-a4f8-d3fc8738954d None admin] PATH_INFO: `/v3/users/b63ef41f5fe6465a923cf4b473a9faa5` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:12.558590 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d499ca59-ed88-4e47-a4f8-d3fc8738954d None admin] RBAC: Authorizing `identity:get_user(user_id=b63ef41f5fe6465a923cf4b473a9faa5)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:12.560048 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d499ca59-ed88-4e47-a4f8-d3fc8738954d None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:12.561607 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 552/1114] 10.222.0.15 () {56 vars in 1177 bytes} [Sun Sep 11 10:35:12 2022] GET /identity/v3/users/b63ef41f5fe6465a923cf4b473a9faa5 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:35:12.567513 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cde19b2-031c-454b-acd4-ef484f1671c2 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:12.664612 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cde19b2-031c-454b-acd4-ef484f1671c2 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:12.699093 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480da97c-6f79-473c-accb-4c65195feb14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:12.699716 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480da97c-6f79-473c-accb-4c65195feb14 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:12.699998 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480da97c-6f79-473c-accb-4c65195feb14 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:12.700271 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480da97c-6f79-473c-accb-4c65195feb14 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:12.726776 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:12.765245 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480da97c-6f79-473c-accb-4c65195feb14 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:12.766689 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480da97c-6f79-473c-accb-4c65195feb14 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:12.860409 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 563/1115] 10.222.0.15 () {58 vars in 1282 bytes} [Sun Sep 11 10:35:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 10:35:12.866470 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d499ca59-ed88-4e47-a4f8-d3fc8738954d None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:12.924473 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:12.964314 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d499ca59-ed88-4e47-a4f8-d3fc8738954d None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:12.998156 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e236b0-1538-4aff-9c8e-d4322cc1ca59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:12.998771 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e236b0-1538-4aff-9c8e-d4322cc1ca59 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:12.999053 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e236b0-1538-4aff-9c8e-d4322cc1ca59 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:12.999325 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e236b0-1538-4aff-9c8e-d4322cc1ca59 None admin] PATH_INFO: `/v3/projects/4117990fba5243fcafb57c9a4a5c16d9` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:13.000445 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e236b0-1538-4aff-9c8e-d4322cc1ca59 None admin] RBAC: Authorizing `identity:get_project(project_id=4117990fba5243fcafb57c9a4a5c16d9)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:13.001859 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e236b0-1538-4aff-9c8e-d4322cc1ca59 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:13.003478 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 553/1116] 10.222.0.15 () {56 vars in 1186 bytes} [Sun Sep 11 10:35:12 2022] GET /identity/v3/projects/4117990fba5243fcafb57c9a4a5c16d9 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:35:13.009481 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480da97c-6f79-473c-accb-4c65195feb14 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:13.066534 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:13.106714 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480da97c-6f79-473c-accb-4c65195feb14 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:13.139111 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe999fbd-d141-4f14-b459-320161f857b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:13.139680 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe999fbd-d141-4f14-b459-320161f857b2 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:13.139964 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe999fbd-d141-4f14-b459-320161f857b2 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:13.140239 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe999fbd-d141-4f14-b459-320161f857b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:13.140949 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe999fbd-d141-4f14-b459-320161f857b2 None admin] RBAC: Adding query filter params (role.id=cdb3aa1bd85b4694964629ff7cf8039b, scope.project.id=4117990fba5243fcafb57c9a4a5c16d9, user.id=b63ef41f5fe6465a923cf4b473a9faa5) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:35:13.141256 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe999fbd-d141-4f14-b459-320161f857b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:13.142683 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe999fbd-d141-4f14-b459-320161f857b2 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:13.148624 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 564/1117] 10.222.0.15 () {56 vars in 1374 bytes} [Sun Sep 11 10:35:13 2022] GET /identity/v3/role_assignments?user.id=b63ef41f5fe6465a923cf4b473a9faa5&scope.project.id=4117990fba5243fcafb57c9a4a5c16d9&role.id=cdb3aa1bd85b4694964629ff7cf8039b => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:35:15.247929 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b731f426-7cab-4ca7-8346-ffbcbc27118d None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:15.248471 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b731f426-7cab-4ca7-8346-ffbcbc27118d None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:15.248471 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b731f426-7cab-4ca7-8346-ffbcbc27118d None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:15.249787 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 554/1118] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:35:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:35:15.254974 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ad921a-f571-4643-9a82-abca6a64f663 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:15.255375 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ad921a-f571-4643-9a82-abca6a64f663 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:15.255536 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ad921a-f571-4643-9a82-abca6a64f663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:15.273732 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-72ad921a-f571-4643-9a82-abca6a64f663 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:35:15.274942 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-72ad921a-f571-4643-9a82-abca6a64f663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:35:15.340788 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 565/1119] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:35:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 10:35:15.615550 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b731f426-7cab-4ca7-8346-ffbcbc27118d None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:15.715822 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b731f426-7cab-4ca7-8346-ffbcbc27118d None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:15.741245 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:15.749019 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4cc931-756f-4600-9c90-4f3436589efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:15.749635 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4cc931-756f-4600-9c90-4f3436589efb None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:15.749913 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4cc931-756f-4600-9c90-4f3436589efb None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:15.750178 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4cc931-756f-4600-9c90-4f3436589efb None admin] PATH_INFO: `/v3/roles/cdb3aa1bd85b4694964629ff7cf8039b` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:15.751558 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4cc931-756f-4600-9c90-4f3436589efb None admin] RBAC: Authorizing `identity:get_role(role_id=cdb3aa1bd85b4694964629ff7cf8039b)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:15.752901 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4cc931-756f-4600-9c90-4f3436589efb None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:15.754183 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 555/1120] 10.222.0.15 () {56 vars in 1177 bytes} [Sun Sep 11 10:35:15 2022] GET /identity/v3/roles/cdb3aa1bd85b4694964629ff7cf8039b => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:35:15.760621 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ad921a-f571-4643-9a82-abca6a64f663 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:15.813816 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:15.858422 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ad921a-f571-4643-9a82-abca6a64f663 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:15.891233 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f19413-d411-4764-b58b-88b5735489af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:15.891817 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f19413-d411-4764-b58b-88b5735489af None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:15.892095 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f19413-d411-4764-b58b-88b5735489af None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:15.892367 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f19413-d411-4764-b58b-88b5735489af None admin] PATH_INFO: `/v3/users/b63ef41f5fe6465a923cf4b473a9faa5` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:15.893466 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f19413-d411-4764-b58b-88b5735489af None admin] RBAC: Authorizing `identity:get_user(user_id=b63ef41f5fe6465a923cf4b473a9faa5)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:15.894893 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f19413-d411-4764-b58b-88b5735489af None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:15.896637 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 566/1121] 10.222.0.15 () {56 vars in 1177 bytes} [Sun Sep 11 10:35:15 2022] GET /identity/v3/users/b63ef41f5fe6465a923cf4b473a9faa5 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:35:15.902417 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4cc931-756f-4600-9c90-4f3436589efb None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:15.999787 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4cc931-756f-4600-9c90-4f3436589efb None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:16.032545 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3115e1d-111f-4744-82b5-1255f5d2bfec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:16.033155 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3115e1d-111f-4744-82b5-1255f5d2bfec None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:16.033431 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3115e1d-111f-4744-82b5-1255f5d2bfec None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:16.033695 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3115e1d-111f-4744-82b5-1255f5d2bfec None admin] PATH_INFO: `/v3/projects/4117990fba5243fcafb57c9a4a5c16d9` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:16.034776 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3115e1d-111f-4744-82b5-1255f5d2bfec None admin] RBAC: Authorizing `identity:get_project(project_id=4117990fba5243fcafb57c9a4a5c16d9)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:16.036995 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3115e1d-111f-4744-82b5-1255f5d2bfec None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:16.038619 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 556/1122] 10.222.0.15 () {56 vars in 1186 bytes} [Sun Sep 11 10:35:15 2022] GET /identity/v3/projects/4117990fba5243fcafb57c9a4a5c16d9 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:35:16.044581 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f19413-d411-4764-b58b-88b5735489af None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:16.141541 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f19413-d411-4764-b58b-88b5735489af None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:16.163862 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:16.175505 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6357c7f-45fa-4ad2-bc04-96c822afcd66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:16.176155 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6357c7f-45fa-4ad2-bc04-96c822afcd66 None admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:16.176436 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6357c7f-45fa-4ad2-bc04-96c822afcd66 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:16.177229 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6357c7f-45fa-4ad2-bc04-96c822afcd66 None admin] PATH_INFO: `/v3/projects/4117990fba5243fcafb57c9a4a5c16d9/users/b63ef41f5fe6465a923cf4b473a9faa5/roles/cdb3aa1bd85b4694964629ff7cf8039b` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:16.179439 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6357c7f-45fa-4ad2-bc04-96c822afcd66 None admin] RBAC: Authorizing `identity:create_grant(project_id=4117990fba5243fcafb57c9a4a5c16d9, user_id=b63ef41f5fe6465a923cf4b473a9faa5, role_id=cdb3aa1bd85b4694964629ff7cf8039b)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:16.180870 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6357c7f-45fa-4ad2-bc04-96c822afcd66 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:16.193546 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 567/1123] 10.222.0.15 () {58 vars in 1439 bytes} [Sun Sep 11 10:35:16 2022] PUT /identity/v3/projects/4117990fba5243fcafb57c9a4a5c16d9/users/b63ef41f5fe6465a923cf4b473a9faa5/roles/cdb3aa1bd85b4694964629ff7cf8039b => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:35:18.333610 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87969227-e700-4e87-9111-4fb594674412 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:18.334049 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87969227-e700-4e87-9111-4fb594674412 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:18.334172 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87969227-e700-4e87-9111-4fb594674412 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:18.335495 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 557/1124] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:35:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:35:18.340772 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256af263-a783-4a8b-bc5c-92b218cc123b None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:18.341143 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256af263-a783-4a8b-bc5c-92b218cc123b None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:18.341330 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256af263-a783-4a8b-bc5c-92b218cc123b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:18.359392 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-256af263-a783-4a8b-bc5c-92b218cc123b None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:35:18.360591 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-256af263-a783-4a8b-bc5c-92b218cc123b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:35:18.424654 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 568/1125] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:35:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 10:35:18.698767 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87969227-e700-4e87-9111-4fb594674412 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:18.760925 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:18.799256 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87969227-e700-4e87-9111-4fb594674412 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:18.831805 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4574ea39-7f05-41c6-a6f0-74c84bd99917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:18.832416 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4574ea39-7f05-41c6-a6f0-74c84bd99917 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:18.833306 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4574ea39-7f05-41c6-a6f0-74c84bd99917 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:18.833820 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4574ea39-7f05-41c6-a6f0-74c84bd99917 None admin] PATH_INFO: `/v3/roles/cdb3aa1bd85b4694964629ff7cf8039b` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:18.835244 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4574ea39-7f05-41c6-a6f0-74c84bd99917 None admin] RBAC: Authorizing `identity:get_role(role_id=cdb3aa1bd85b4694964629ff7cf8039b)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:18.836627 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4574ea39-7f05-41c6-a6f0-74c84bd99917 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:18.837910 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 558/1126] 10.222.0.15 () {56 vars in 1177 bytes} [Sun Sep 11 10:35:18 2022] GET /identity/v3/roles/cdb3aa1bd85b4694964629ff7cf8039b => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:35:18.843640 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256af263-a783-4a8b-bc5c-92b218cc123b None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:18.939675 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256af263-a783-4a8b-bc5c-92b218cc123b None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:18.952892 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:18.974630 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9d8474-4700-40be-af2d-4fb6818aa4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:18.975246 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9d8474-4700-40be-af2d-4fb6818aa4ac None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:18.975552 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9d8474-4700-40be-af2d-4fb6818aa4ac None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:18.975824 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9d8474-4700-40be-af2d-4fb6818aa4ac None admin] PATH_INFO: `/v3/users/b63ef41f5fe6465a923cf4b473a9faa5` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:18.976908 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9d8474-4700-40be-af2d-4fb6818aa4ac None admin] RBAC: Authorizing `identity:get_user(user_id=b63ef41f5fe6465a923cf4b473a9faa5)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:18.978579 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9d8474-4700-40be-af2d-4fb6818aa4ac None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:18.980153 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 569/1127] 10.222.0.15 () {56 vars in 1177 bytes} [Sun Sep 11 10:35:18 2022] GET /identity/v3/users/b63ef41f5fe6465a923cf4b473a9faa5 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:35:18.985846 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4574ea39-7f05-41c6-a6f0-74c84bd99917 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:19.086907 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4574ea39-7f05-41c6-a6f0-74c84bd99917 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:19.115815 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:19.121116 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f3a4f8-8152-418c-bbad-074acf90c314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:19.121712 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f3a4f8-8152-418c-bbad-074acf90c314 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:19.121992 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f3a4f8-8152-418c-bbad-074acf90c314 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:19.122263 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f3a4f8-8152-418c-bbad-074acf90c314 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:19.187618 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f3a4f8-8152-418c-bbad-074acf90c314 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:19.189034 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f3a4f8-8152-418c-bbad-074acf90c314 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:19.283391 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 559/1128] 10.222.0.15 () {58 vars in 1282 bytes} [Sun Sep 11 10:35:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 10:35:19.289759 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9d8474-4700-40be-af2d-4fb6818aa4ac None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:19.386121 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9d8474-4700-40be-af2d-4fb6818aa4ac None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:19.418440 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7f15e5-afad-4d70-94f9-22562f08ce94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:19.419137 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f15e5-afad-4d70-94f9-22562f08ce94 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:19.419449 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f15e5-afad-4d70-94f9-22562f08ce94 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:19.419859 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f15e5-afad-4d70-94f9-22562f08ce94 None admin] PATH_INFO: `/v3/projects/4117990fba5243fcafb57c9a4a5c16d9` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:19.420942 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7f15e5-afad-4d70-94f9-22562f08ce94 None admin] RBAC: Authorizing `identity:get_project(project_id=4117990fba5243fcafb57c9a4a5c16d9)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:19.422351 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7f15e5-afad-4d70-94f9-22562f08ce94 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:19.423974 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 570/1129] 10.222.0.15 () {56 vars in 1186 bytes} [Sun Sep 11 10:35:19 2022] GET /identity/v3/projects/4117990fba5243fcafb57c9a4a5c16d9 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:35:19.430297 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f3a4f8-8152-418c-bbad-074acf90c314 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:19.460792 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:19.527766 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f3a4f8-8152-418c-bbad-074acf90c314 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:19.560701 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ace676c-f2df-4bcd-90b0-6deec49e3ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:19.561233 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ace676c-f2df-4bcd-90b0-6deec49e3ea4 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:19.561514 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ace676c-f2df-4bcd-90b0-6deec49e3ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:19.561788 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ace676c-f2df-4bcd-90b0-6deec49e3ea4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:19.562501 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ace676c-f2df-4bcd-90b0-6deec49e3ea4 None admin] RBAC: Adding query filter params (role.id=cdb3aa1bd85b4694964629ff7cf8039b, scope.project.id=4117990fba5243fcafb57c9a4a5c16d9, user.id=b63ef41f5fe6465a923cf4b473a9faa5) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:35:19.562807 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ace676c-f2df-4bcd-90b0-6deec49e3ea4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:19.564237 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ace676c-f2df-4bcd-90b0-6deec49e3ea4 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:19.570434 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 560/1130] 10.222.0.15 () {56 vars in 1374 bytes} [Sun Sep 11 10:35:19 2022] GET /identity/v3/role_assignments?user.id=b63ef41f5fe6465a923cf4b473a9faa5&scope.project.id=4117990fba5243fcafb57c9a4a5c16d9&role.id=cdb3aa1bd85b4694964629ff7cf8039b => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:35:21.723884 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d029d6-ec45-4a1e-9c6e-1db1a8e72304 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:21.724676 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d029d6-ec45-4a1e-9c6e-1db1a8e72304 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:21.724676 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d029d6-ec45-4a1e-9c6e-1db1a8e72304 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:21.725735 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 571/1131] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:35:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:35:21.731138 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b041f8-5d25-4ec7-bbec-7c81fa697a83 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:21.731540 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b041f8-5d25-4ec7-bbec-7c81fa697a83 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:21.731708 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b041f8-5d25-4ec7-bbec-7c81fa697a83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:21.750124 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-f1b041f8-5d25-4ec7-bbec-7c81fa697a83 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:35:21.751307 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-f1b041f8-5d25-4ec7-bbec-7c81fa697a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:35:21.816662 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 561/1132] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:35:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 10:35:22.085786 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d029d6-ec45-4a1e-9c6e-1db1a8e72304 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:22.135176 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:22.185616 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d029d6-ec45-4a1e-9c6e-1db1a8e72304 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:22.218191 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe627b7-d221-4d46-9d6e-e55fc99ccfaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:22.218815 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe627b7-d221-4d46-9d6e-e55fc99ccfaf None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:22.219122 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe627b7-d221-4d46-9d6e-e55fc99ccfaf None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:22.219395 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe627b7-d221-4d46-9d6e-e55fc99ccfaf None admin] PATH_INFO: `/v3/domains/default` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:22.220534 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe627b7-d221-4d46-9d6e-e55fc99ccfaf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:22.221951 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe627b7-d221-4d46-9d6e-e55fc99ccfaf None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:22.223672 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 572/1133] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:35:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:35:22.229741 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b041f8-5d25-4ec7-bbec-7c81fa697a83 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:22.256501 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:22.327095 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b041f8-5d25-4ec7-bbec-7c81fa697a83 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:22.359643 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74676978-ef13-40f8-8ed3-cd8f3a7f9c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:22.360179 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74676978-ef13-40f8-8ed3-cd8f3a7f9c8a None admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:22.360455 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74676978-ef13-40f8-8ed3-cd8f3a7f9c8a None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:22.360723 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74676978-ef13-40f8-8ed3-cd8f3a7f9c8a None admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:22.361360 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74676978-ef13-40f8-8ed3-cd8f3a7f9c8a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:22.362773 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74676978-ef13-40f8-8ed3-cd8f3a7f9c8a None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:22.390171 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 562/1134] 10.222.0.15 () {60 vars in 1141 bytes} [Sun Sep 11 10:35:22 2022] POST /identity/v3/projects => generated 311 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:35:24.540166 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb2de96-7df9-439f-93d3-65eea6aac69d None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:24.540691 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb2de96-7df9-439f-93d3-65eea6aac69d None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:24.540691 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb2de96-7df9-439f-93d3-65eea6aac69d None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:24.542030 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 573/1135] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:35:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:35:24.547287 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d6f860-2fe8-4332-bbc9-0511daed7bfa None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:24.547769 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d6f860-2fe8-4332-bbc9-0511daed7bfa None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:24.547864 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d6f860-2fe8-4332-bbc9-0511daed7bfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:24.566117 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-f7d6f860-2fe8-4332-bbc9-0511daed7bfa None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:35:24.567889 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-f7d6f860-2fe8-4332-bbc9-0511daed7bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:35:24.632857 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 563/1136] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:35:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 10:35:24.901683 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb2de96-7df9-439f-93d3-65eea6aac69d None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:24.999268 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb2de96-7df9-439f-93d3-65eea6aac69d None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:25.015496 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:25.033133 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4d599a-b849-4dd9-8c6b-8c9cde079339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:25.033750 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4d599a-b849-4dd9-8c6b-8c9cde079339 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:25.034030 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4d599a-b849-4dd9-8c6b-8c9cde079339 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:25.034300 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4d599a-b849-4dd9-8c6b-8c9cde079339 None admin] PATH_INFO: `/v3/domains/default` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:25.035381 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4d599a-b849-4dd9-8c6b-8c9cde079339 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:25.036932 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4d599a-b849-4dd9-8c6b-8c9cde079339 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:25.038479 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 574/1137] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:35:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:35:25.044118 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d6f860-2fe8-4332-bbc9-0511daed7bfa None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:25.054573 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:25.142645 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d6f860-2fe8-4332-bbc9-0511daed7bfa None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:25.175050 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0ac20b-b3e9-46e7-bc5b-c34c07838cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:25.175619 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ac20b-b3e9-46e7-bc5b-c34c07838cee None admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:25.175905 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ac20b-b3e9-46e7-bc5b-c34c07838cee None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:25.176182 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ac20b-b3e9-46e7-bc5b-c34c07838cee None admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:25.177356 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0ac20b-b3e9-46e7-bc5b-c34c07838cee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:25.178773 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0ac20b-b3e9-46e7-bc5b-c34c07838cee None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:25.199643 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 564/1138] 10.222.0.15 () {60 vars in 1132 bytes} [Sun Sep 11 10:35:25 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 10:35:27.393004 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920f5896-415e-41bc-bbe8-9f1b87060d44 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:27.393481 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920f5896-415e-41bc-bbe8-9f1b87060d44 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:27.393573 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920f5896-415e-41bc-bbe8-9f1b87060d44 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:27.395401 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 575/1139] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:35:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:35:27.400944 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713c9f9e-3c0c-4ecd-b885-2f7ba404290b None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:27.401331 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713c9f9e-3c0c-4ecd-b885-2f7ba404290b None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:27.401489 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713c9f9e-3c0c-4ecd-b885-2f7ba404290b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:27.420067 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-713c9f9e-3c0c-4ecd-b885-2f7ba404290b None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:35:27.421238 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-713c9f9e-3c0c-4ecd-b885-2f7ba404290b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:35:27.479585 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:27.489554 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 565/1140] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:35:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 10:35:27.768672 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920f5896-415e-41bc-bbe8-9f1b87060d44 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:27.868360 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920f5896-415e-41bc-bbe8-9f1b87060d44 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:27.900442 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8840ead4-fb8a-4b3e-8d9e-bcb62c515062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:27.901067 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8840ead4-fb8a-4b3e-8d9e-bcb62c515062 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:27.901345 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8840ead4-fb8a-4b3e-8d9e-bcb62c515062 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:27.901617 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8840ead4-fb8a-4b3e-8d9e-bcb62c515062 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:27.912077 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8840ead4-fb8a-4b3e-8d9e-bcb62c515062 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:35:27.916778 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-8840ead4-fb8a-4b3e-8d9e-bcb62c515062 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 10:35:27.916778 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:35:27.916778 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:35:27.916778 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:35:27.916778 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:35:27.916778 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:35:27.916778 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:35:27.916778 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:35:27.916778 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:35:27.916778 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:35:27.916778 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:35:27.916778 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:35:27.916778 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:35:27.916778 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:35:27.916778 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:35:27.916778 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:35:27.917894 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:35:27.917894 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:35:27.917894 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:35:27.917894 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:35:27.917894 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:35:27.917894 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:35:27.917894 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:35:27.917894 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:35:27.917894 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:35:27.917894 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:35:27.917894 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:35:27.917894 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:35:27.917894 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:35:27.917894 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:35:27.917894 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:35:27.917894 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:35:27.918961 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:35:27.918961 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:35:27.918961 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:35:27.918961 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:35:27.918961 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:35:27.918961 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:35:27.918961 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:35:27.918961 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:35:27.918961 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:35:27.918961 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:35:27.918961 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 10:35:27.918961 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:35:27.918961 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 576/1141] 10.222.0.15 () {56 vars in 1096 bytes} [Sun Sep 11 10:35:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:35:27.923921 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713c9f9e-3c0c-4ecd-b885-2f7ba404290b None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:28.019126 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713c9f9e-3c0c-4ecd-b885-2f7ba404290b None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:28.052743 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd25a257-bb90-4256-ad3a-6a9638dca5d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:28.053328 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd25a257-bb90-4256-ad3a-6a9638dca5d0 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:28.053604 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd25a257-bb90-4256-ad3a-6a9638dca5d0 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:28.053867 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd25a257-bb90-4256-ad3a-6a9638dca5d0 None admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:28.054508 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd25a257-bb90-4256-ad3a-6a9638dca5d0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:35:28.054804 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd25a257-bb90-4256-ad3a-6a9638dca5d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:28.056174 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd25a257-bb90-4256-ad3a-6a9638dca5d0 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:28.063774 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 566/1142] 10.222.0.15 () {56 vars in 1099 bytes} [Sun Sep 11 10:35:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:35:28.070134 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8840ead4-fb8a-4b3e-8d9e-bcb62c515062 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:28.115241 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:28.167798 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8840ead4-fb8a-4b3e-8d9e-bcb62c515062 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:28.200757 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a90685-1fc7-44c9-a08c-cb6261fa42fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:28.201368 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a90685-1fc7-44c9-a08c-cb6261fa42fc None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:28.201643 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a90685-1fc7-44c9-a08c-cb6261fa42fc None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:28.201912 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a90685-1fc7-44c9-a08c-cb6261fa42fc None admin] PATH_INFO: `/v3/users/5766825638b64b40989ad3339598c5cc` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:28.215516 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a90685-1fc7-44c9-a08c-cb6261fa42fc None admin] RBAC: Authorizing `identity:get_user(user_id=5766825638b64b40989ad3339598c5cc)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:28.216942 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a90685-1fc7-44c9-a08c-cb6261fa42fc None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:28.218494 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 577/1143] 10.222.0.15 () {56 vars in 1177 bytes} [Sun Sep 11 10:35:28 2022] GET /identity/v3/users/5766825638b64b40989ad3339598c5cc => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:35:28.224339 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd25a257-bb90-4256-ad3a-6a9638dca5d0 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:28.281308 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:28.324975 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd25a257-bb90-4256-ad3a-6a9638dca5d0 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:28.357844 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf32eac-1f41-4cef-823b-c14bcb137160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:28.358444 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf32eac-1f41-4cef-823b-c14bcb137160 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:28.358725 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf32eac-1f41-4cef-823b-c14bcb137160 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:28.358997 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf32eac-1f41-4cef-823b-c14bcb137160 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:28.424101 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf32eac-1f41-4cef-823b-c14bcb137160 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:28.425534 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf32eac-1f41-4cef-823b-c14bcb137160 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:28.478037 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:28.521425 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 567/1144] 10.222.0.15 () {58 vars in 1282 bytes} [Sun Sep 11 10:35:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 10:35:28.527323 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a90685-1fc7-44c9-a08c-cb6261fa42fc None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:28.624195 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a90685-1fc7-44c9-a08c-cb6261fa42fc None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:28.625599 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:28.658645 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6424ee5-3937-439d-8c5c-7b2cde93c159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:28.659260 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6424ee5-3937-439d-8c5c-7b2cde93c159 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:28.659569 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6424ee5-3937-439d-8c5c-7b2cde93c159 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:28.659842 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6424ee5-3937-439d-8c5c-7b2cde93c159 None admin] PATH_INFO: `/v3/projects/d4c33536a4b445b89ffc0457c5e658d4` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:28.661159 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6424ee5-3937-439d-8c5c-7b2cde93c159 None admin] RBAC: Authorizing `identity:get_project(project_id=d4c33536a4b445b89ffc0457c5e658d4)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:28.662575 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6424ee5-3937-439d-8c5c-7b2cde93c159 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:28.664200 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 578/1145] 10.222.0.15 () {56 vars in 1186 bytes} [Sun Sep 11 10:35:28 2022] GET /identity/v3/projects/d4c33536a4b445b89ffc0457c5e658d4 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:35:28.670231 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf32eac-1f41-4cef-823b-c14bcb137160 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:28.765390 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf32eac-1f41-4cef-823b-c14bcb137160 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:28.798134 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4396f1-0947-4736-9e7f-f1904fec2aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:28.798725 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4396f1-0947-4736-9e7f-f1904fec2aab None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:28.799009 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4396f1-0947-4736-9e7f-f1904fec2aab None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:28.799278 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4396f1-0947-4736-9e7f-f1904fec2aab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:28.800021 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4396f1-0947-4736-9e7f-f1904fec2aab None admin] RBAC: Adding query filter params (role.id=2cb7d73f0ea8414296367573f6902340, scope.project.id=d4c33536a4b445b89ffc0457c5e658d4, user.id=5766825638b64b40989ad3339598c5cc) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:35:28.800322 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4396f1-0947-4736-9e7f-f1904fec2aab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:28.802268 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4396f1-0947-4736-9e7f-f1904fec2aab None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:28.807395 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 568/1146] 10.222.0.15 () {56 vars in 1374 bytes} [Sun Sep 11 10:35:28 2022] GET /identity/v3/role_assignments?user.id=5766825638b64b40989ad3339598c5cc&scope.project.id=d4c33536a4b445b89ffc0457c5e658d4&role.id=2cb7d73f0ea8414296367573f6902340 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:35:30.915252 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6077686-0053-4d6a-8bc7-bfaf0613bde3 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:30.915855 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6077686-0053-4d6a-8bc7-bfaf0613bde3 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:30.915855 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6077686-0053-4d6a-8bc7-bfaf0613bde3 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:30.917188 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 579/1147] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:35:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:35:30.922259 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de04f4de-2e04-47ed-9418-088a3bb9b53f None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:30.922646 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de04f4de-2e04-47ed-9418-088a3bb9b53f None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:30.922813 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de04f4de-2e04-47ed-9418-088a3bb9b53f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:30.941329 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-de04f4de-2e04-47ed-9418-088a3bb9b53f None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:35:30.942686 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-de04f4de-2e04-47ed-9418-088a3bb9b53f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:35:30.956245 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:31.008054 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 569/1148] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:35:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 10:35:31.277360 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6077686-0053-4d6a-8bc7-bfaf0613bde3 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:31.376337 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6077686-0053-4d6a-8bc7-bfaf0613bde3 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:31.408917 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2389bb26-779c-4669-9b55-3270f73a401c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:31.409535 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2389bb26-779c-4669-9b55-3270f73a401c None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:31.409813 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2389bb26-779c-4669-9b55-3270f73a401c None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:31.410083 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2389bb26-779c-4669-9b55-3270f73a401c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:31.420467 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2389bb26-779c-4669-9b55-3270f73a401c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:35:31.425206 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-2389bb26-779c-4669-9b55-3270f73a401c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 10:35:31.425206 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:35:31.425206 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:35:31.425206 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:35:31.425206 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:35:31.425206 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:35:31.425206 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:35:31.425206 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:35:31.425206 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:35:31.425206 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:35:31.425206 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:35:31.425206 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:35:31.425206 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:35:31.425206 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:35:31.425206 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:35:31.425206 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:35:31.426393 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:35:31.426393 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:35:31.426393 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:35:31.426393 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:35:31.426393 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:35:31.426393 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:35:31.426393 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:35:31.426393 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:35:31.426393 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:35:31.426393 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:35:31.426393 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:35:31.426393 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:35:31.426393 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:35:31.426393 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:35:31.426393 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:35:31.426393 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:35:31.427581 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:35:31.427581 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:35:31.427581 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:35:31.427581 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:35:31.427581 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:35:31.427581 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:35:31.427581 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:35:31.427581 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:35:31.427581 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:35:31.427581 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:35:31.427581 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 10:35:31.427581 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:35:31.427581 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 580/1149] 10.222.0.15 () {56 vars in 1096 bytes} [Sun Sep 11 10:35:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:35:31.432497 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de04f4de-2e04-47ed-9418-088a3bb9b53f None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:31.527573 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de04f4de-2e04-47ed-9418-088a3bb9b53f None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:31.559735 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997a010e-7fc5-435e-8464-9803645e3829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:31.560331 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a010e-7fc5-435e-8464-9803645e3829 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:31.560608 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a010e-7fc5-435e-8464-9803645e3829 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:31.560875 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a010e-7fc5-435e-8464-9803645e3829 None admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:31.561525 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997a010e-7fc5-435e-8464-9803645e3829 None admin] RBAC: Adding query filter params (name=admin) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:35:31.561836 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997a010e-7fc5-435e-8464-9803645e3829 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:31.563184 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997a010e-7fc5-435e-8464-9803645e3829 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:31.571694 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 570/1150] 10.222.0.15 () {56 vars in 1099 bytes} [Sun Sep 11 10:35:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:35:31.577555 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2389bb26-779c-4669-9b55-3270f73a401c None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:31.608734 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:31.676921 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2389bb26-779c-4669-9b55-3270f73a401c None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:31.710120 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc213459-9715-4d60-a78d-b0dfbecd9ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:31.710735 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc213459-9715-4d60-a78d-b0dfbecd9ade None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:31.711010 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc213459-9715-4d60-a78d-b0dfbecd9ade None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:31.711275 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc213459-9715-4d60-a78d-b0dfbecd9ade None admin] PATH_INFO: `/v3/users/5766825638b64b40989ad3339598c5cc` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:31.712383 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc213459-9715-4d60-a78d-b0dfbecd9ade None admin] RBAC: Authorizing `identity:get_user(user_id=5766825638b64b40989ad3339598c5cc)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:31.713792 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc213459-9715-4d60-a78d-b0dfbecd9ade None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:31.715324 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 581/1151] 10.222.0.15 () {56 vars in 1177 bytes} [Sun Sep 11 10:35:31 2022] GET /identity/v3/users/5766825638b64b40989ad3339598c5cc => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:35:31.721284 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997a010e-7fc5-435e-8464-9803645e3829 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:31.737794 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:31.820401 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997a010e-7fc5-435e-8464-9803645e3829 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:31.853181 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45426da6-cc98-4f43-a712-59327d147d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:31.853795 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45426da6-cc98-4f43-a712-59327d147d44 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:31.854074 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45426da6-cc98-4f43-a712-59327d147d44 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:31.854342 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45426da6-cc98-4f43-a712-59327d147d44 None admin] PATH_INFO: `/v3/projects/d4c33536a4b445b89ffc0457c5e658d4` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:31.855457 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45426da6-cc98-4f43-a712-59327d147d44 None admin] RBAC: Authorizing `identity:get_project(project_id=d4c33536a4b445b89ffc0457c5e658d4)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:31.856875 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45426da6-cc98-4f43-a712-59327d147d44 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:31.858462 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 571/1152] 10.222.0.15 () {56 vars in 1186 bytes} [Sun Sep 11 10:35:31 2022] GET /identity/v3/projects/d4c33536a4b445b89ffc0457c5e658d4 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:35:31.864603 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc213459-9715-4d60-a78d-b0dfbecd9ade None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:31.958601 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:31.962144 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc213459-9715-4d60-a78d-b0dfbecd9ade None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:31.995685 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7f748a-e587-4681-b054-e8ec96f54a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:31.996335 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7f748a-e587-4681-b054-e8ec96f54a33 None admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:31.996619 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7f748a-e587-4681-b054-e8ec96f54a33 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:31.996893 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7f748a-e587-4681-b054-e8ec96f54a33 None admin] PATH_INFO: `/v3/projects/d4c33536a4b445b89ffc0457c5e658d4/users/5766825638b64b40989ad3339598c5cc/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:31.998742 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7f748a-e587-4681-b054-e8ec96f54a33 None admin] RBAC: Authorizing `identity:create_grant(project_id=d4c33536a4b445b89ffc0457c5e658d4, user_id=5766825638b64b40989ad3339598c5cc, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:32.000184 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7f748a-e587-4681-b054-e8ec96f54a33 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:32.013644 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 582/1153] 10.222.0.15 () {58 vars in 1439 bytes} [Sun Sep 11 10:35:31 2022] PUT /identity/v3/projects/d4c33536a4b445b89ffc0457c5e658d4/users/5766825638b64b40989ad3339598c5cc/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:35:34.139585 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7d4876-1730-4b8c-ac5a-2ab67eca7742 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:34.140145 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7d4876-1730-4b8c-ac5a-2ab67eca7742 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:34.140145 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7d4876-1730-4b8c-ac5a-2ab67eca7742 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:34.141461 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 572/1154] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:35:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:35:34.147111 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcac5ae-a9cb-4c01-9cb4-ea66f5b68648 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:34.147570 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcac5ae-a9cb-4c01-9cb4-ea66f5b68648 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:34.147670 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcac5ae-a9cb-4c01-9cb4-ea66f5b68648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:34.166101 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-4fcac5ae-a9cb-4c01-9cb4-ea66f5b68648 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:35:34.167273 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-4fcac5ae-a9cb-4c01-9cb4-ea66f5b68648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:35:34.231786 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 583/1155] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:35:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 10:35:34.502145 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7d4876-1730-4b8c-ac5a-2ab67eca7742 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:34.582620 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:34.603124 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7d4876-1730-4b8c-ac5a-2ab67eca7742 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:34.635868 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7280abdb-c7fd-47c4-b6c4-845ec326bfc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:34.636485 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7280abdb-c7fd-47c4-b6c4-845ec326bfc1 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:34.636764 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7280abdb-c7fd-47c4-b6c4-845ec326bfc1 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:34.637035 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7280abdb-c7fd-47c4-b6c4-845ec326bfc1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:34.647651 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7280abdb-c7fd-47c4-b6c4-845ec326bfc1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:35:34.652371 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-7280abdb-c7fd-47c4-b6c4-845ec326bfc1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 10:35:34.652371 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:35:34.652371 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:35:34.652371 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:35:34.652371 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:35:34.652371 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:35:34.652371 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:35:34.652371 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:35:34.652371 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:35:34.652371 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:35:34.652371 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:35:34.652371 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:35:34.652371 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:35:34.652371 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:35:34.652371 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:35:34.652371 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:35:34.653465 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:35:34.653465 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:35:34.653465 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:35:34.653465 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:35:34.653465 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:35:34.653465 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:35:34.653465 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:35:34.653465 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:35:34.653465 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:35:34.653465 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:35:34.653465 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:35:34.653465 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:35:34.653465 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:35:34.653465 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:35:34.653465 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:35:34.653465 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:35:34.654471 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:35:34.654471 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:35:34.654471 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:35:34.654471 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:35:34.654471 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:35:34.654471 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:35:34.654471 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:35:34.654471 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:35:34.654471 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:35:34.654471 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:35:34.654471 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 10:35:34.654471 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:35:34.654471 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 573/1156] 10.222.0.15 () {56 vars in 1096 bytes} [Sun Sep 11 10:35:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:35:34.659507 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcac5ae-a9cb-4c01-9cb4-ea66f5b68648 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:34.745025 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:34.757671 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcac5ae-a9cb-4c01-9cb4-ea66f5b68648 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:34.790168 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c0d263-2fe9-40eb-bac8-cc118fbf7784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:34.790767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c0d263-2fe9-40eb-bac8-cc118fbf7784 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:34.791044 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c0d263-2fe9-40eb-bac8-cc118fbf7784 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:34.791313 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c0d263-2fe9-40eb-bac8-cc118fbf7784 None admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:34.792003 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c0d263-2fe9-40eb-bac8-cc118fbf7784 None admin] RBAC: Adding query filter params (name=admin) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:35:34.792305 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c0d263-2fe9-40eb-bac8-cc118fbf7784 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:34.793675 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c0d263-2fe9-40eb-bac8-cc118fbf7784 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:34.802052 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 584/1157] 10.222.0.15 () {56 vars in 1099 bytes} [Sun Sep 11 10:35:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:35:34.807965 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7280abdb-c7fd-47c4-b6c4-845ec326bfc1 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:34.905028 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7280abdb-c7fd-47c4-b6c4-845ec326bfc1 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:34.937794 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a77aa9c-10d6-456d-a7e2-1bdd401ef279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:34.938411 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a77aa9c-10d6-456d-a7e2-1bdd401ef279 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:34.938723 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a77aa9c-10d6-456d-a7e2-1bdd401ef279 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:34.938999 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a77aa9c-10d6-456d-a7e2-1bdd401ef279 None admin] PATH_INFO: `/v3/users/5766825638b64b40989ad3339598c5cc` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:34.940145 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a77aa9c-10d6-456d-a7e2-1bdd401ef279 None admin] RBAC: Authorizing `identity:get_user(user_id=5766825638b64b40989ad3339598c5cc)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:34.942153 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a77aa9c-10d6-456d-a7e2-1bdd401ef279 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:34.943950 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 574/1158] 10.222.0.15 () {56 vars in 1177 bytes} [Sun Sep 11 10:35:34 2022] GET /identity/v3/users/5766825638b64b40989ad3339598c5cc => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:35:34.949570 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c0d263-2fe9-40eb-bac8-cc118fbf7784 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:35.045707 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c0d263-2fe9-40eb-bac8-cc118fbf7784 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:35.080155 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb76c70-f3df-409b-81ae-e18fd93c23e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:35.080746 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb76c70-f3df-409b-81ae-e18fd93c23e4 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:35.081022 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb76c70-f3df-409b-81ae-e18fd93c23e4 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:35.081292 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb76c70-f3df-409b-81ae-e18fd93c23e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:35.099276 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:35.147459 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb76c70-f3df-409b-81ae-e18fd93c23e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:35.148905 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb76c70-f3df-409b-81ae-e18fd93c23e4 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:35.243575 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 585/1159] 10.222.0.15 () {58 vars in 1282 bytes} [Sun Sep 11 10:35:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 10:35:35.249581 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a77aa9c-10d6-456d-a7e2-1bdd401ef279 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:35.260116 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:35.348674 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a77aa9c-10d6-456d-a7e2-1bdd401ef279 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:35.381940 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89afd836-c635-44f1-8d36-d5e13c312212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:35.382559 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89afd836-c635-44f1-8d36-d5e13c312212 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:35.382838 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89afd836-c635-44f1-8d36-d5e13c312212 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:35.383110 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89afd836-c635-44f1-8d36-d5e13c312212 None admin] PATH_INFO: `/v3/projects/d4c33536a4b445b89ffc0457c5e658d4` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:35.384244 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89afd836-c635-44f1-8d36-d5e13c312212 None admin] RBAC: Authorizing `identity:get_project(project_id=d4c33536a4b445b89ffc0457c5e658d4)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:35.385665 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89afd836-c635-44f1-8d36-d5e13c312212 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:35.387258 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 575/1160] 10.222.0.15 () {56 vars in 1186 bytes} [Sun Sep 11 10:35:35 2022] GET /identity/v3/projects/d4c33536a4b445b89ffc0457c5e658d4 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:35:35.393414 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb76c70-f3df-409b-81ae-e18fd93c23e4 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:35.441531 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:35.490069 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb76c70-f3df-409b-81ae-e18fd93c23e4 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:35.523101 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbebbdd-eb05-4b0f-99ad-efc3fbc8b009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:35.523767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbebbdd-eb05-4b0f-99ad-efc3fbc8b009 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:35.524050 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbebbdd-eb05-4b0f-99ad-efc3fbc8b009 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:35.524320 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbebbdd-eb05-4b0f-99ad-efc3fbc8b009 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:35.525025 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbebbdd-eb05-4b0f-99ad-efc3fbc8b009 None admin] RBAC: Adding query filter params (role.id=2cb7d73f0ea8414296367573f6902340, scope.project.id=d4c33536a4b445b89ffc0457c5e658d4, user.id=5766825638b64b40989ad3339598c5cc) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:35:35.525330 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbebbdd-eb05-4b0f-99ad-efc3fbc8b009 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:35.526737 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbebbdd-eb05-4b0f-99ad-efc3fbc8b009 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:35.532332 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 586/1161] 10.222.0.15 () {56 vars in 1374 bytes} [Sun Sep 11 10:35:35 2022] GET /identity/v3/role_assignments?user.id=5766825638b64b40989ad3339598c5cc&scope.project.id=d4c33536a4b445b89ffc0457c5e658d4&role.id=2cb7d73f0ea8414296367573f6902340 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:35:37.715032 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eefe272-4a32-466e-9404-6ad197a03e08 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:37.715582 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eefe272-4a32-466e-9404-6ad197a03e08 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:37.715582 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eefe272-4a32-466e-9404-6ad197a03e08 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:37.716856 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 576/1162] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:35:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:35:37.722352 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e1c9e-20f0-499d-804f-c95b84dcb3b5 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:37.723630 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e1c9e-20f0-499d-804f-c95b84dcb3b5 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:37.723630 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e1c9e-20f0-499d-804f-c95b84dcb3b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:37.742037 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-9e9e1c9e-20f0-499d-804f-c95b84dcb3b5 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:35:37.743219 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-9e9e1c9e-20f0-499d-804f-c95b84dcb3b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:35:37.807271 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 587/1163] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:35:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 10:35:38.086974 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eefe272-4a32-466e-9404-6ad197a03e08 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:38.163720 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:38.187314 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eefe272-4a32-466e-9404-6ad197a03e08 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:38.220086 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11c40a6-b528-4bf3-8ed5-0f5c1caee3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:38.220685 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11c40a6-b528-4bf3-8ed5-0f5c1caee3bf None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:38.220963 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11c40a6-b528-4bf3-8ed5-0f5c1caee3bf None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:38.221232 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11c40a6-b528-4bf3-8ed5-0f5c1caee3bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:38.287684 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11c40a6-b528-4bf3-8ed5-0f5c1caee3bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:38.289123 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11c40a6-b528-4bf3-8ed5-0f5c1caee3bf None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:38.361660 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:38.385157 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 577/1164] 10.222.0.15 () {58 vars in 1282 bytes} [Sun Sep 11 10:35:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 10:35:38.391205 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9e1c9e-20f0-499d-804f-c95b84dcb3b5 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:38.435025 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:38.488515 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9e1c9e-20f0-499d-804f-c95b84dcb3b5 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:38.522559 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba4290d-5abd-4a3f-bf90-564189782534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:38.523179 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba4290d-5abd-4a3f-bf90-564189782534 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:38.523484 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba4290d-5abd-4a3f-bf90-564189782534 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:38.523763 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba4290d-5abd-4a3f-bf90-564189782534 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:38.530241 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba4290d-5abd-4a3f-bf90-564189782534 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:38.531720 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba4290d-5abd-4a3f-bf90-564189782534 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:38.543013 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-fba4290d-5abd-4a3f-bf90-564189782534 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 11 10:35:38.543013 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:35:38.543013 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 11 10:35:38.543013 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 11 10:35:38.543013 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 10:35:38.543013 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 10:35:38.543013 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 10:35:38.543013 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 10:35:38.543013 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 11 10:35:38.543013 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:35:38.543013 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 10:35:38.543013 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:35:38.543013 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:35:38.543013 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:35:38.543013 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:35:38.543013 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:35:38.543013 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:35:38.544192 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:35:38.544192 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:35:38.544192 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:35:38.544192 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:35:38.544192 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:35:38.544192 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:35:38.544192 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:35:38.544192 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:35:38.544192 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:35:38.544192 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:35:38.544192 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:35:38.544192 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:35:38.544192 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:35:38.544192 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:35:38.544192 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:35:38.544192 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:35:38.545114 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:35:38.545114 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:35:38.545114 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:35:38.545114 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:35:38.545114 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:35:38.545114 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:35:38.545114 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:35:38.545114 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:35:38.545114 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:35:38.545114 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:35:38.545114 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 11 10:35:38.545114 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:35:38.545114 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 11 10:35:38.545114 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:35:38.545114 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 588/1165] 10.222.0.15 () {56 vars in 1117 bytes} [Sun Sep 11 10:35:38 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:35:38.550182 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11c40a6-b528-4bf3-8ed5-0f5c1caee3bf None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:38.647169 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11c40a6-b528-4bf3-8ed5-0f5c1caee3bf None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:38.680280 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02a6ebf-7c86-4707-b311-fbca3abcfb0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:38.680874 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a6ebf-7c86-4707-b311-fbca3abcfb0c None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:38.681152 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a6ebf-7c86-4707-b311-fbca3abcfb0c None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:38.681421 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a6ebf-7c86-4707-b311-fbca3abcfb0c None admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:38.682048 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02a6ebf-7c86-4707-b311-fbca3abcfb0c None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:35:38.682345 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02a6ebf-7c86-4707-b311-fbca3abcfb0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:38.683718 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02a6ebf-7c86-4707-b311-fbca3abcfb0c None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:38.690692 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 578/1166] 10.222.0.15 () {56 vars in 1115 bytes} [Sun Sep 11 10:35:38 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:35:40.802998 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5efbbb-acfc-40a2-9628-e33f6f892fa9 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:40.803573 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5efbbb-acfc-40a2-9628-e33f6f892fa9 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:40.803573 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5efbbb-acfc-40a2-9628-e33f6f892fa9 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:40.804881 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 589/1167] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:35:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:35:40.810065 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c97c67-aaed-44f4-89b9-d9e153bbb19a None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:40.810453 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c97c67-aaed-44f4-89b9-d9e153bbb19a None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:40.810618 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c97c67-aaed-44f4-89b9-d9e153bbb19a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:40.828337 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-51c97c67-aaed-44f4-89b9-d9e153bbb19a None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:35:40.829508 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-51c97c67-aaed-44f4-89b9-d9e153bbb19a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:35:40.863291 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:40.896181 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 579/1168] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:35:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 10:35:41.166555 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5efbbb-acfc-40a2-9628-e33f6f892fa9 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:41.264597 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5efbbb-acfc-40a2-9628-e33f6f892fa9 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:41.266791 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:41.298265 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8374b8-cb7a-4368-bc95-ef078161edb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:41.298863 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8374b8-cb7a-4368-bc95-ef078161edb8 None admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:41.299139 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8374b8-cb7a-4368-bc95-ef078161edb8 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:41.299406 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8374b8-cb7a-4368-bc95-ef078161edb8 None admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:41.300076 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8374b8-cb7a-4368-bc95-ef078161edb8 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:41.301416 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8374b8-cb7a-4368-bc95-ef078161edb8 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:41.318329 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 590/1169] 10.222.0.15 () {60 vars in 1138 bytes} [Sun Sep 11 10:35:41 2022] POST /identity/v3/domains => generated 267 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:35:43.510931 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e4f1d-104d-454a-9f88-6511a05530c8 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:43.511467 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e4f1d-104d-454a-9f88-6511a05530c8 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:43.511589 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e4f1d-104d-454a-9f88-6511a05530c8 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:43.512840 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 580/1170] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:35:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:35:43.518208 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75384d7d-1de5-4d88-84b6-6d9639f3931b None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:43.518573 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75384d7d-1de5-4d88-84b6-6d9639f3931b None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:43.518756 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75384d7d-1de5-4d88-84b6-6d9639f3931b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:43.537410 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-75384d7d-1de5-4d88-84b6-6d9639f3931b None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:35:43.538579 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-75384d7d-1de5-4d88-84b6-6d9639f3931b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:35:43.605282 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 591/1171] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:35:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 10:35:43.875252 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e4f1d-104d-454a-9f88-6511a05530c8 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:43.975494 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e4f1d-104d-454a-9f88-6511a05530c8 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:44.009635 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9fc11a-8ae6-4bee-b8b4-6ede5f3fb9b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:44.010255 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9fc11a-8ae6-4bee-b8b4-6ede5f3fb9b9 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:44.010534 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9fc11a-8ae6-4bee-b8b4-6ede5f3fb9b9 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:44.010808 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9fc11a-8ae6-4bee-b8b4-6ede5f3fb9b9 None admin] PATH_INFO: `/v3/domains/68f81fb9834040c2a4ce2900751011bf` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:44.020913 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9fc11a-8ae6-4bee-b8b4-6ede5f3fb9b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=68f81fb9834040c2a4ce2900751011bf)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:44.022348 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9fc11a-8ae6-4bee-b8b4-6ede5f3fb9b9 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:44.023940 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 581/1172] 10.222.0.15 () {56 vars in 1183 bytes} [Sun Sep 11 10:35:43 2022] GET /identity/v3/domains/68f81fb9834040c2a4ce2900751011bf => generated 267 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:35:44.029727 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75384d7d-1de5-4d88-84b6-6d9639f3931b None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:44.118342 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:44.126137 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75384d7d-1de5-4d88-84b6-6d9639f3931b None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:44.158761 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2674ca13-4e06-4643-81b6-facb07bc2006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:44.159360 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2674ca13-4e06-4643-81b6-facb07bc2006 None admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:44.159666 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2674ca13-4e06-4643-81b6-facb07bc2006 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:44.159937 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2674ca13-4e06-4643-81b6-facb07bc2006 None admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:44.160582 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2674ca13-4e06-4643-81b6-facb07bc2006 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:44.162586 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2674ca13-4e06-4643-81b6-facb07bc2006 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:44.188055 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 592/1173] 10.222.0.15 () {60 vars in 1141 bytes} [Sun Sep 11 10:35:44 2022] POST /identity/v3/projects => generated 361 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:35:46.330405 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d48de0d-8f80-4fee-8917-9b40b90f7af2 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:46.330815 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d48de0d-8f80-4fee-8917-9b40b90f7af2 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:46.330950 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d48de0d-8f80-4fee-8917-9b40b90f7af2 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:46.332309 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 582/1174] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:35:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:35:46.338335 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc27d2de-d892-4c77-9050-4db0664eb672 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:46.338769 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc27d2de-d892-4c77-9050-4db0664eb672 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:46.338892 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc27d2de-d892-4c77-9050-4db0664eb672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:46.357496 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-dc27d2de-d892-4c77-9050-4db0664eb672 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:35:46.358663 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-dc27d2de-d892-4c77-9050-4db0664eb672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:35:46.422521 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 593/1175] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:35:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 10:35:46.696999 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d48de0d-8f80-4fee-8917-9b40b90f7af2 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:46.727500 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:46.800155 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d48de0d-8f80-4fee-8917-9b40b90f7af2 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:46.833231 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf8025d-96ee-47bb-b665-58374fcf0277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:46.833844 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf8025d-96ee-47bb-b665-58374fcf0277 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:46.834123 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf8025d-96ee-47bb-b665-58374fcf0277 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:46.834395 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf8025d-96ee-47bb-b665-58374fcf0277 None admin] PATH_INFO: `/v3/domains/68f81fb9834040c2a4ce2900751011bf` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:46.835520 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf8025d-96ee-47bb-b665-58374fcf0277 None admin] RBAC: Authorizing `identity:get_domain(domain_id=68f81fb9834040c2a4ce2900751011bf)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:46.836945 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf8025d-96ee-47bb-b665-58374fcf0277 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:46.838483 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 583/1176] 10.222.0.15 () {56 vars in 1183 bytes} [Sun Sep 11 10:35:46 2022] GET /identity/v3/domains/68f81fb9834040c2a4ce2900751011bf => generated 267 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:35:46.844198 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc27d2de-d892-4c77-9050-4db0664eb672 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:46.918661 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:46.941475 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc27d2de-d892-4c77-9050-4db0664eb672 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:46.975369 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a811fe-a17e-488f-a0cc-1c269a7adfe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:46.975935 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a811fe-a17e-488f-a0cc-1c269a7adfe3 None admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:46.976215 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a811fe-a17e-488f-a0cc-1c269a7adfe3 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:46.976487 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a811fe-a17e-488f-a0cc-1c269a7adfe3 None admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:46.977116 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a811fe-a17e-488f-a0cc-1c269a7adfe3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:46.978791 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a811fe-a17e-488f-a0cc-1c269a7adfe3 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:46.999805 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 594/1177] 10.222.0.15 () {60 vars in 1132 bytes} [Sun Sep 11 10:35:46 2022] POST /identity/v3/users => generated 311 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:35:49.165591 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c2c98-c9f5-407b-b779-d7b3b73bc4c5 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:49.166242 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c2c98-c9f5-407b-b779-d7b3b73bc4c5 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:49.166242 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c2c98-c9f5-407b-b779-d7b3b73bc4c5 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:49.167472 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 584/1178] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:35:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:35:49.172682 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b87ee7-59e4-4fbc-89fd-194a6f972ce3 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:49.173103 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b87ee7-59e4-4fbc-89fd-194a6f972ce3 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:49.173209 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b87ee7-59e4-4fbc-89fd-194a6f972ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:49.191611 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-42b87ee7-59e4-4fbc-89fd-194a6f972ce3 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:35:49.193522 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-42b87ee7-59e4-4fbc-89fd-194a6f972ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:35:49.257338 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 595/1179] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:35:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 10:35:49.528612 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642c2c98-c9f5-407b-b779-d7b3b73bc4c5 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:49.629397 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:49.635407 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642c2c98-c9f5-407b-b779-d7b3b73bc4c5 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:49.669912 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb33231-5371-4eef-8eb0-6e9d73407b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:49.670728 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb33231-5371-4eef-8eb0-6e9d73407b50 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:49.670953 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb33231-5371-4eef-8eb0-6e9d73407b50 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:49.671281 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb33231-5371-4eef-8eb0-6e9d73407b50 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:49.682857 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb33231-5371-4eef-8eb0-6e9d73407b50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:35:49.687706 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-4eb33231-5371-4eef-8eb0-6e9d73407b50 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 10:35:49.687706 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:35:49.687706 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:35:49.687706 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:35:49.687706 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:35:49.687706 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:35:49.687706 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:35:49.687706 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:35:49.687706 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:35:49.687706 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:35:49.687706 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:35:49.687706 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:35:49.687706 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:35:49.687706 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:35:49.687706 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:35:49.687706 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:35:49.688918 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:35:49.688918 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:35:49.688918 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:35:49.688918 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:35:49.688918 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:35:49.688918 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:35:49.688918 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:35:49.688918 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:35:49.688918 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:35:49.688918 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:35:49.688918 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:35:49.688918 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:35:49.688918 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:35:49.688918 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:35:49.688918 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:35:49.688918 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:35:49.689914 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:35:49.689914 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:35:49.689914 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:35:49.689914 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:35:49.689914 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:35:49.689914 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:35:49.689914 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:35:49.689914 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:35:49.689914 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:35:49.689914 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:35:49.689914 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 10:35:49.689914 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:35:49.693847 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 585/1180] 10.222.0.15 () {56 vars in 1096 bytes} [Sun Sep 11 10:35:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:35:49.696480 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b87ee7-59e4-4fbc-89fd-194a6f972ce3 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:49.759104 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:49.797417 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b87ee7-59e4-4fbc-89fd-194a6f972ce3 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:49.830771 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d94bfa-9e8f-45a6-8265-5fff4684668a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:49.831374 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d94bfa-9e8f-45a6-8265-5fff4684668a None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:49.831683 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d94bfa-9e8f-45a6-8265-5fff4684668a None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:49.831955 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d94bfa-9e8f-45a6-8265-5fff4684668a None admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:49.832733 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d94bfa-9e8f-45a6-8265-5fff4684668a None admin] RBAC: Adding query filter params (name=admin) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:35:49.833040 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d94bfa-9e8f-45a6-8265-5fff4684668a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:49.834406 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d94bfa-9e8f-45a6-8265-5fff4684668a None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:49.842196 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 596/1181] 10.222.0.15 () {56 vars in 1099 bytes} [Sun Sep 11 10:35:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:35:49.849219 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb33231-5371-4eef-8eb0-6e9d73407b50 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:49.948106 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb33231-5371-4eef-8eb0-6e9d73407b50 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:49.980791 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ba64fb-af68-4c24-b989-c4e7542c858b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:49.981393 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ba64fb-af68-4c24-b989-c4e7542c858b None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:49.981668 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ba64fb-af68-4c24-b989-c4e7542c858b None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:49.981934 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ba64fb-af68-4c24-b989-c4e7542c858b None admin] PATH_INFO: `/v3/users/ea05b51b5f274948b0f73a6770dc2bab` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:49.995575 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ba64fb-af68-4c24-b989-c4e7542c858b None admin] RBAC: Authorizing `identity:get_user(user_id=ea05b51b5f274948b0f73a6770dc2bab)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:50.001670 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ba64fb-af68-4c24-b989-c4e7542c858b None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:50.003298 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 586/1182] 10.222.0.15 () {56 vars in 1177 bytes} [Sun Sep 11 10:35:49 2022] GET /identity/v3/users/ea05b51b5f274948b0f73a6770dc2bab => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:35:50.009849 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d94bfa-9e8f-45a6-8265-5fff4684668a None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:50.105655 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d94bfa-9e8f-45a6-8265-5fff4684668a None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:50.127909 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:50.139736 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0ed01b-9fff-438d-8d37-4c7b71442c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:50.140341 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0ed01b-9fff-438d-8d37-4c7b71442c7a None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:50.140645 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0ed01b-9fff-438d-8d37-4c7b71442c7a None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:50.140918 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0ed01b-9fff-438d-8d37-4c7b71442c7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:50.205514 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0ed01b-9fff-438d-8d37-4c7b71442c7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:50.206956 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0ed01b-9fff-438d-8d37-4c7b71442c7a None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:50.302624 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 597/1183] 10.222.0.15 () {58 vars in 1282 bytes} [Sun Sep 11 10:35:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 10:35:50.308762 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ba64fb-af68-4c24-b989-c4e7542c858b None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:50.326434 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:50.405467 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ba64fb-af68-4c24-b989-c4e7542c858b None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:50.438573 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f62358b-ace9-4b96-9e73-00a79de8b04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:50.439186 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f62358b-ace9-4b96-9e73-00a79de8b04d None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:50.439486 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f62358b-ace9-4b96-9e73-00a79de8b04d None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:50.439760 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f62358b-ace9-4b96-9e73-00a79de8b04d None admin] PATH_INFO: `/v3/projects/e3a89927e5ff48238ddce5fa77017d2c` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:50.440859 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f62358b-ace9-4b96-9e73-00a79de8b04d None admin] RBAC: Authorizing `identity:get_project(project_id=e3a89927e5ff48238ddce5fa77017d2c)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:50.442479 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f62358b-ace9-4b96-9e73-00a79de8b04d None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:50.444111 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 587/1184] 10.222.0.15 () {56 vars in 1186 bytes} [Sun Sep 11 10:35:50 2022] GET /identity/v3/projects/e3a89927e5ff48238ddce5fa77017d2c => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:35:50.450161 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0ed01b-9fff-438d-8d37-4c7b71442c7a None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:50.474398 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:50.547281 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0ed01b-9fff-438d-8d37-4c7b71442c7a None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:50.580447 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b208467-9438-4b96-a1d3-7ea9939d0f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:50.581027 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b208467-9438-4b96-a1d3-7ea9939d0f96 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:50.581307 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b208467-9438-4b96-a1d3-7ea9939d0f96 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:50.581578 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b208467-9438-4b96-a1d3-7ea9939d0f96 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:50.582286 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b208467-9438-4b96-a1d3-7ea9939d0f96 None admin] RBAC: Adding query filter params (role.id=2cb7d73f0ea8414296367573f6902340, scope.project.id=e3a89927e5ff48238ddce5fa77017d2c, user.id=ea05b51b5f274948b0f73a6770dc2bab) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:35:50.582726 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b208467-9438-4b96-a1d3-7ea9939d0f96 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:50.584391 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b208467-9438-4b96-a1d3-7ea9939d0f96 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:50.589544 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 598/1185] 10.222.0.15 () {56 vars in 1374 bytes} [Sun Sep 11 10:35:50 2022] GET /identity/v3/role_assignments?user.id=ea05b51b5f274948b0f73a6770dc2bab&scope.project.id=e3a89927e5ff48238ddce5fa77017d2c&role.id=2cb7d73f0ea8414296367573f6902340 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:35:52.709522 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceee3a1-e302-4b2f-8669-0cf892c80a2e None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:52.710031 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceee3a1-e302-4b2f-8669-0cf892c80a2e None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:52.710031 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceee3a1-e302-4b2f-8669-0cf892c80a2e None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:52.711310 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 588/1186] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:35:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:35:52.716640 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a030c3ad-a627-4ed5-b249-635ad5f978ea None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:52.717023 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a030c3ad-a627-4ed5-b249-635ad5f978ea None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:52.717185 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a030c3ad-a627-4ed5-b249-635ad5f978ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:52.735795 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-a030c3ad-a627-4ed5-b249-635ad5f978ea None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:35:52.736954 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-a030c3ad-a627-4ed5-b249-635ad5f978ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:35:52.800585 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 599/1187] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:35:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 10:35:53.072395 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceee3a1-e302-4b2f-8669-0cf892c80a2e None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:53.156754 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:53.173332 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceee3a1-e302-4b2f-8669-0cf892c80a2e None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:53.206403 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1216f17a-3c94-442c-b362-4e7d9709325f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:53.207022 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1216f17a-3c94-442c-b362-4e7d9709325f None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:53.207305 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1216f17a-3c94-442c-b362-4e7d9709325f None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:53.207771 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1216f17a-3c94-442c-b362-4e7d9709325f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:53.217578 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1216f17a-3c94-442c-b362-4e7d9709325f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:35:53.222399 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-1216f17a-3c94-442c-b362-4e7d9709325f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 10:35:53.222399 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:35:53.222399 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:35:53.222399 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:35:53.222399 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:35:53.222399 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:35:53.222399 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:35:53.222399 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:35:53.222399 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:35:53.222399 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:35:53.222399 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:35:53.222399 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:35:53.222399 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:35:53.222399 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:35:53.222399 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:35:53.222399 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:35:53.224199 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:35:53.224199 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:35:53.224199 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:35:53.224199 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:35:53.224199 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:35:53.224199 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:35:53.224199 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:35:53.224199 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:35:53.224199 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:35:53.224199 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:35:53.224199 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:35:53.224199 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:35:53.224199 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:35:53.224199 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:35:53.224199 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:35:53.224199 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:35:53.225215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:35:53.225215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:35:53.225215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:35:53.225215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:35:53.225215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:35:53.225215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:35:53.225215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:35:53.225215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:35:53.225215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:35:53.225215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:35:53.225215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 10:35:53.225215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:35:53.225215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 589/1188] 10.222.0.15 () {56 vars in 1096 bytes} [Sun Sep 11 10:35:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:35:53.230551 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a030c3ad-a627-4ed5-b249-635ad5f978ea None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:53.248919 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:53.327353 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a030c3ad-a627-4ed5-b249-635ad5f978ea None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:53.360065 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5538b416-dad5-4d62-990f-e264e668b98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:53.360661 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538b416-dad5-4d62-990f-e264e668b98d None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:53.360936 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538b416-dad5-4d62-990f-e264e668b98d None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:53.361206 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538b416-dad5-4d62-990f-e264e668b98d None admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:53.361852 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5538b416-dad5-4d62-990f-e264e668b98d None admin] RBAC: Adding query filter params (name=admin) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:35:53.362147 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5538b416-dad5-4d62-990f-e264e668b98d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:53.363533 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5538b416-dad5-4d62-990f-e264e668b98d None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:53.371939 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 600/1189] 10.222.0.15 () {56 vars in 1099 bytes} [Sun Sep 11 10:35:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:35:53.377856 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1216f17a-3c94-442c-b362-4e7d9709325f None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:53.474899 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1216f17a-3c94-442c-b362-4e7d9709325f None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:53.508439 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a929e059-f951-4c76-b892-fee25ddfa2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:53.509045 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a929e059-f951-4c76-b892-fee25ddfa2ec None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:53.509324 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a929e059-f951-4c76-b892-fee25ddfa2ec None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:53.509595 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a929e059-f951-4c76-b892-fee25ddfa2ec None admin] PATH_INFO: `/v3/users/ea05b51b5f274948b0f73a6770dc2bab` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:53.510687 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a929e059-f951-4c76-b892-fee25ddfa2ec None admin] RBAC: Authorizing `identity:get_user(user_id=ea05b51b5f274948b0f73a6770dc2bab)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:53.512131 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a929e059-f951-4c76-b892-fee25ddfa2ec None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:53.513677 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 590/1190] 10.222.0.15 () {56 vars in 1177 bytes} [Sun Sep 11 10:35:53 2022] GET /identity/v3/users/ea05b51b5f274948b0f73a6770dc2bab => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:35:53.519281 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5538b416-dad5-4d62-990f-e264e668b98d None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:53.597709 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:53.617042 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5538b416-dad5-4d62-990f-e264e668b98d None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:53.649212 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb9ba04-b34b-45e8-8cef-daa086d8379e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:53.649828 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb9ba04-b34b-45e8-8cef-daa086d8379e None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:53.650108 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb9ba04-b34b-45e8-8cef-daa086d8379e None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:53.650380 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb9ba04-b34b-45e8-8cef-daa086d8379e None admin] PATH_INFO: `/v3/projects/e3a89927e5ff48238ddce5fa77017d2c` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:53.651498 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb9ba04-b34b-45e8-8cef-daa086d8379e None admin] RBAC: Authorizing `identity:get_project(project_id=e3a89927e5ff48238ddce5fa77017d2c)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:53.652922 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb9ba04-b34b-45e8-8cef-daa086d8379e None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:53.654531 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 601/1191] 10.222.0.15 () {56 vars in 1186 bytes} [Sun Sep 11 10:35:53 2022] GET /identity/v3/projects/e3a89927e5ff48238ddce5fa77017d2c => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:35:53.660400 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a929e059-f951-4c76-b892-fee25ddfa2ec None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:53.675084 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:53.759224 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a929e059-f951-4c76-b892-fee25ddfa2ec None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:53.791586 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71859179-8e0e-4ded-ad2c-79688e3440cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:53.792231 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71859179-8e0e-4ded-ad2c-79688e3440cc None admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:53.792511 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71859179-8e0e-4ded-ad2c-79688e3440cc None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:53.792783 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71859179-8e0e-4ded-ad2c-79688e3440cc None admin] PATH_INFO: `/v3/projects/e3a89927e5ff48238ddce5fa77017d2c/users/ea05b51b5f274948b0f73a6770dc2bab/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:53.794663 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71859179-8e0e-4ded-ad2c-79688e3440cc None admin] RBAC: Authorizing `identity:create_grant(project_id=e3a89927e5ff48238ddce5fa77017d2c, user_id=ea05b51b5f274948b0f73a6770dc2bab, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:53.796120 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71859179-8e0e-4ded-ad2c-79688e3440cc None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:53.808689 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 591/1192] 10.222.0.15 () {58 vars in 1439 bytes} [Sun Sep 11 10:35:53 2022] PUT /identity/v3/projects/e3a89927e5ff48238ddce5fa77017d2c/users/ea05b51b5f274948b0f73a6770dc2bab/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:35:55.919378 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4a6bcf-9455-472d-93cf-eb8474479695 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:55.919957 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4a6bcf-9455-472d-93cf-eb8474479695 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:55.919957 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4a6bcf-9455-472d-93cf-eb8474479695 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:55.921301 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 602/1193] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:35:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:35:55.926956 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49356e1-e456-4842-b18e-c84753682e9e None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:55.927388 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49356e1-e456-4842-b18e-c84753682e9e None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:55.927616 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49356e1-e456-4842-b18e-c84753682e9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:55.945878 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-b49356e1-e456-4842-b18e-c84753682e9e None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:35:55.947049 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-b49356e1-e456-4842-b18e-c84753682e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:35:56.011360 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 592/1194] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:35:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 10:35:56.280843 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4a6bcf-9455-472d-93cf-eb8474479695 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:56.379693 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4a6bcf-9455-472d-93cf-eb8474479695 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:56.411931 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddf1631-42aa-481e-aaab-06ffc592f8a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:56.412540 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf1631-42aa-481e-aaab-06ffc592f8a5 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:56.412818 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf1631-42aa-481e-aaab-06ffc592f8a5 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:56.413086 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf1631-42aa-481e-aaab-06ffc592f8a5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:56.422745 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddf1631-42aa-481e-aaab-06ffc592f8a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:35:56.428383 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-9ddf1631-42aa-481e-aaab-06ffc592f8a5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 10:35:56.428383 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:35:56.428383 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:35:56.428383 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:35:56.428383 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:35:56.428383 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:35:56.428383 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:35:56.428383 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:35:56.428383 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:35:56.428383 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:35:56.428383 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:35:56.428383 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:35:56.428383 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:35:56.428383 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:35:56.428383 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:35:56.428383 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:35:56.429504 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:35:56.429504 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:35:56.429504 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:35:56.429504 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:35:56.429504 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:35:56.429504 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:35:56.429504 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:35:56.429504 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:35:56.429504 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:35:56.429504 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:35:56.429504 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:35:56.429504 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:35:56.429504 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:35:56.429504 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:35:56.429504 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:35:56.429504 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:35:56.430354 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:35:56.430354 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:35:56.430354 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:35:56.430354 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:35:56.430354 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:35:56.430354 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:35:56.430354 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:35:56.430354 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:35:56.430354 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:35:56.430354 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:35:56.430354 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 10:35:56.430354 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:35:56.430354 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 603/1195] 10.222.0.15 () {56 vars in 1096 bytes} [Sun Sep 11 10:35:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:35:56.435449 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49356e1-e456-4842-b18e-c84753682e9e None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:56.437653 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:56.533735 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49356e1-e456-4842-b18e-c84753682e9e None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:56.566893 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6bd816-cd0b-4830-922f-b14bdd220369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:56.567592 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6bd816-cd0b-4830-922f-b14bdd220369 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:56.567872 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6bd816-cd0b-4830-922f-b14bdd220369 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:56.568144 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6bd816-cd0b-4830-922f-b14bdd220369 None admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:56.568794 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6bd816-cd0b-4830-922f-b14bdd220369 None admin] RBAC: Adding query filter params (name=admin) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:35:56.569095 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6bd816-cd0b-4830-922f-b14bdd220369 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:56.570449 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6bd816-cd0b-4830-922f-b14bdd220369 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:56.578135 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 593/1196] 10.222.0.15 () {56 vars in 1099 bytes} [Sun Sep 11 10:35:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:35:56.585024 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddf1631-42aa-481e-aaab-06ffc592f8a5 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:56.595983 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:56.683271 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddf1631-42aa-481e-aaab-06ffc592f8a5 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:56.716207 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d58bac5-6777-4806-82fa-4a6ee6a701a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:56.716824 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d58bac5-6777-4806-82fa-4a6ee6a701a6 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:56.717106 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d58bac5-6777-4806-82fa-4a6ee6a701a6 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:56.717377 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d58bac5-6777-4806-82fa-4a6ee6a701a6 None admin] PATH_INFO: `/v3/users/ea05b51b5f274948b0f73a6770dc2bab` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:56.718475 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d58bac5-6777-4806-82fa-4a6ee6a701a6 None admin] RBAC: Authorizing `identity:get_user(user_id=ea05b51b5f274948b0f73a6770dc2bab)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:56.719936 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d58bac5-6777-4806-82fa-4a6ee6a701a6 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:56.721492 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 604/1197] 10.222.0.15 () {56 vars in 1177 bytes} [Sun Sep 11 10:35:56 2022] GET /identity/v3/users/ea05b51b5f274948b0f73a6770dc2bab => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:35:56.728133 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6bd816-cd0b-4830-922f-b14bdd220369 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:56.788528 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:56.825436 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6bd816-cd0b-4830-922f-b14bdd220369 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:56.857919 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b2c071-89d4-458a-8dff-3338ec04ad19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:56.858516 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b2c071-89d4-458a-8dff-3338ec04ad19 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:56.858795 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b2c071-89d4-458a-8dff-3338ec04ad19 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:56.859067 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b2c071-89d4-458a-8dff-3338ec04ad19 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:56.924181 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b2c071-89d4-458a-8dff-3338ec04ad19 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:56.925606 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b2c071-89d4-458a-8dff-3338ec04ad19 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:56.983568 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:57.020622 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 594/1198] 10.222.0.15 () {58 vars in 1282 bytes} [Sun Sep 11 10:35:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 10:35:57.026530 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d58bac5-6777-4806-82fa-4a6ee6a701a6 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:57.103282 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:35:57.126523 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d58bac5-6777-4806-82fa-4a6ee6a701a6 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:57.159201 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aef8ec-4655-4c4b-bc8f-f5fcde6b56f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:57.159852 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aef8ec-4655-4c4b-bc8f-f5fcde6b56f5 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:57.160131 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aef8ec-4655-4c4b-bc8f-f5fcde6b56f5 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:57.160403 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aef8ec-4655-4c4b-bc8f-f5fcde6b56f5 None admin] PATH_INFO: `/v3/projects/e3a89927e5ff48238ddce5fa77017d2c` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:57.161499 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aef8ec-4655-4c4b-bc8f-f5fcde6b56f5 None admin] RBAC: Authorizing `identity:get_project(project_id=e3a89927e5ff48238ddce5fa77017d2c)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:57.162932 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aef8ec-4655-4c4b-bc8f-f5fcde6b56f5 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:57.164589 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 605/1199] 10.222.0.15 () {56 vars in 1186 bytes} [Sun Sep 11 10:35:57 2022] GET /identity/v3/projects/e3a89927e5ff48238ddce5fa77017d2c => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:35:57.170783 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b2c071-89d4-458a-8dff-3338ec04ad19 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:35:57.265619 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b2c071-89d4-458a-8dff-3338ec04ad19 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:35:57.299021 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7968f27b-56be-413b-94da-2633cbe74ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:35:57.299641 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7968f27b-56be-413b-94da-2633cbe74ed5 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:35:57.299921 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7968f27b-56be-413b-94da-2633cbe74ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:35:57.300190 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7968f27b-56be-413b-94da-2633cbe74ed5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:35:57.300896 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7968f27b-56be-413b-94da-2633cbe74ed5 None admin] RBAC: Adding query filter params (role.id=2cb7d73f0ea8414296367573f6902340, scope.project.id=e3a89927e5ff48238ddce5fa77017d2c, user.id=ea05b51b5f274948b0f73a6770dc2bab) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:35:57.301759 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7968f27b-56be-413b-94da-2633cbe74ed5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:35:57.303260 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7968f27b-56be-413b-94da-2633cbe74ed5 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:35:57.308815 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 595/1200] 10.222.0.15 () {56 vars in 1374 bytes} [Sun Sep 11 10:35:57 2022] GET /identity/v3/role_assignments?user.id=ea05b51b5f274948b0f73a6770dc2bab&scope.project.id=e3a89927e5ff48238ddce5fa77017d2c&role.id=2cb7d73f0ea8414296367573f6902340 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:36:07.155110 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=28 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:36:07.306244 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=86 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:36:53.139328 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723bb6a1-05a0-4643-96c4-bf8a7690cbae None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:36:53.139968 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723bb6a1-05a0-4643-96c4-bf8a7690cbae None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:36:53.139968 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723bb6a1-05a0-4643-96c4-bf8a7690cbae None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:36:53.141261 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 606/1201] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:36:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:36:53.147281 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef1cff1-fc9a-4666-bd79-1ae2a8ec7c48 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:36:53.147843 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef1cff1-fc9a-4666-bd79-1ae2a8ec7c48 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:36:53.147843 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef1cff1-fc9a-4666-bd79-1ae2a8ec7c48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:36:53.168471 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-5ef1cff1-fc9a-4666-bd79-1ae2a8ec7c48 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:36:53.169663 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-5ef1cff1-fc9a-4666-bd79-1ae2a8ec7c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:36:53.256219 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 596/1202] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:36:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 10:36:53.531166 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723bb6a1-05a0-4643-96c4-bf8a7690cbae None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:36:53.672250 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723bb6a1-05a0-4643-96c4-bf8a7690cbae None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:36:53.718540 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74d5ce8-e55c-4c54-bee6-9f526d7b4062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:36:53.719170 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d5ce8-e55c-4c54-bee6-9f526d7b4062 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:36:53.719485 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d5ce8-e55c-4c54-bee6-9f526d7b4062 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:36:53.719761 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d5ce8-e55c-4c54-bee6-9f526d7b4062 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:36:53.734246 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74d5ce8-e55c-4c54-bee6-9f526d7b4062 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:36:53.735749 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74d5ce8-e55c-4c54-bee6-9f526d7b4062 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:36:53.751914 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-e74d5ce8-e55c-4c54-bee6-9f526d7b4062 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:36:53.751914 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:36:53.751914 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:36:53.751914 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:36:53.751914 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:36:53.751914 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:36:53.751914 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:36:53.751914 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:36:53.751914 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:36:53.751914 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:36:53.751914 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:36:53.751914 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:36:53.751914 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:36:53.751914 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:36:53.751914 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:36:53.751914 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:36:53.753412 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:36:53.753412 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:36:53.753412 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:36:53.753412 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:36:53.753412 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:36:53.753412 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:36:53.753412 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:36:53.753412 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:36:53.753412 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:36:53.753412 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:36:53.753412 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:36:53.753412 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:36:53.753412 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:36:53.753412 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:36:53.753412 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:36:53.753412 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:36:53.753412 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:36:53.754409 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:36:53.754409 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:36:53.754409 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:36:53.754409 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 607/1203] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:36:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:36:53.759538 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef1cff1-fc9a-4666-bd79-1ae2a8ec7c48 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:36:53.892566 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef1cff1-fc9a-4666-bd79-1ae2a8ec7c48 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:36:53.928844 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:36:53.935288 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4f8225-95d3-4fa5-af59-be71c1f83b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:36:53.936087 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4f8225-95d3-4fa5-af59-be71c1f83b68 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:36:53.936620 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4f8225-95d3-4fa5-af59-be71c1f83b68 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:36:53.936928 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4f8225-95d3-4fa5-af59-be71c1f83b68 None admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:36:53.937651 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4f8225-95d3-4fa5-af59-be71c1f83b68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:36:53.937907 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4f8225-95d3-4fa5-af59-be71c1f83b68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:36:53.939342 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4f8225-95d3-4fa5-af59-be71c1f83b68 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:36:53.957996 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 597/1204] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:36:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:36:53.964367 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74d5ce8-e55c-4c54-bee6-9f526d7b4062 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:36:54.030514 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:36:54.091139 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74d5ce8-e55c-4c54-bee6-9f526d7b4062 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:36:54.136388 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8f9a87-a0db-43a7-96f1-8a0b089ab44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:36:54.137031 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8f9a87-a0db-43a7-96f1-8a0b089ab44a None admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:36:54.137312 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8f9a87-a0db-43a7-96f1-8a0b089ab44a None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:36:54.137602 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8f9a87-a0db-43a7-96f1-8a0b089ab44a None admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:36:54.138258 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8f9a87-a0db-43a7-96f1-8a0b089ab44a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:36:54.139760 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8f9a87-a0db-43a7-96f1-8a0b089ab44a None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:36:54.170174 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 608/1205] 10.222.0.15 () {60 vars in 1132 bytes} [Sun Sep 11 10:36:53 2022] POST /identity/v3/users => generated 251 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:36:56.993828 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd95f0be-59ae-45be-b82d-ec02c6f94af6 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:36:56.994417 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd95f0be-59ae-45be-b82d-ec02c6f94af6 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:36:56.994417 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd95f0be-59ae-45be-b82d-ec02c6f94af6 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:36:56.996256 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 598/1206] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:36:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:36:57.002352 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97263dce-2718-45b2-a42c-34426a8a704e None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:36:57.003166 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97263dce-2718-45b2-a42c-34426a8a704e None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:36:57.003654 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97263dce-2718-45b2-a42c-34426a8a704e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:36:57.032413 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-97263dce-2718-45b2-a42c-34426a8a704e None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:36:57.034940 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-97263dce-2718-45b2-a42c-34426a8a704e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:36:57.118670 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 609/1207] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:36:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 10:36:57.425299 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd95f0be-59ae-45be-b82d-ec02c6f94af6 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:36:57.559774 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd95f0be-59ae-45be-b82d-ec02c6f94af6 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:36:57.602183 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c6479a-0cc3-4d7a-8ddc-21ab5d747728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:36:57.602740 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c6479a-0cc3-4d7a-8ddc-21ab5d747728 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:36:57.603018 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c6479a-0cc3-4d7a-8ddc-21ab5d747728 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:36:57.603585 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c6479a-0cc3-4d7a-8ddc-21ab5d747728 None admin] PATH_INFO: `/v3/roles/service` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:36:57.614525 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c6479a-0cc3-4d7a-8ddc-21ab5d747728 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:36:57.619294 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-01c6479a-0cc3-4d7a-8ddc-21ab5d747728 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 10:36:57.619294 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:36:57.619294 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:36:57.619294 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:36:57.619294 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:36:57.619294 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:36:57.619294 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:36:57.619294 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:36:57.619294 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:36:57.619294 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:36:57.619294 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:36:57.619294 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:36:57.619294 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:36:57.619294 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:36:57.619294 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:36:57.619294 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:36:57.620601 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:36:57.620601 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:36:57.620601 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:36:57.620601 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:36:57.620601 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:36:57.620601 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:36:57.620601 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:36:57.620601 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:36:57.620601 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:36:57.620601 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:36:57.620601 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:36:57.620601 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:36:57.620601 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:36:57.620601 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:36:57.620601 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:36:57.620601 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:36:57.621548 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:36:57.621548 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:36:57.621548 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:36:57.621548 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:36:57.621548 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:36:57.621548 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:36:57.621548 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:36:57.621548 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:36:57.621548 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:36:57.621548 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:36:57.621548 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 10:36:57.621548 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:36:57.621548 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 599/1208] 10.222.0.15 () {56 vars in 1102 bytes} [Sun Sep 11 10:36:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:36:57.626852 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97263dce-2718-45b2-a42c-34426a8a704e None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:36:57.669542 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:36:57.747852 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97263dce-2718-45b2-a42c-34426a8a704e None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:36:57.786659 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12940ef-1d0f-45a6-9d95-9ba1d19c54c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:36:57.787685 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12940ef-1d0f-45a6-9d95-9ba1d19c54c1 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:36:57.788251 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12940ef-1d0f-45a6-9d95-9ba1d19c54c1 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:36:57.788735 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12940ef-1d0f-45a6-9d95-9ba1d19c54c1 None admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:36:57.789651 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12940ef-1d0f-45a6-9d95-9ba1d19c54c1 None admin] RBAC: Adding query filter params (name=service) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:36:57.790178 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12940ef-1d0f-45a6-9d95-9ba1d19c54c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:36:57.791794 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12940ef-1d0f-45a6-9d95-9ba1d19c54c1 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:36:57.805834 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 610/1209] 10.222.0.15 () {56 vars in 1103 bytes} [Sun Sep 11 10:36:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:36:57.807011 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c6479a-0cc3-4d7a-8ddc-21ab5d747728 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:36:57.864664 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:36:57.920045 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c6479a-0cc3-4d7a-8ddc-21ab5d747728 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:36:57.962855 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ee6ed-dc1c-4afb-804c-705d265a7c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:36:57.963532 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ee6ed-dc1c-4afb-804c-705d265a7c6a None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:36:57.963817 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ee6ed-dc1c-4afb-804c-705d265a7c6a None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:36:57.964108 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ee6ed-dc1c-4afb-804c-705d265a7c6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:36:57.975452 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1ee6ed-dc1c-4afb-804c-705d265a7c6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:36:57.976905 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1ee6ed-dc1c-4afb-804c-705d265a7c6a None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:36:57.994215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-4e1ee6ed-dc1c-4afb-804c-705d265a7c6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:36:57.994215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:36:57.994215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:36:57.994215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:36:57.994215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:36:57.994215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:36:57.994215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:36:57.994215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:36:57.994215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:36:57.994215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:36:57.994215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:36:57.994215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:36:57.994215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:36:57.994215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:36:57.994215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:36:57.994215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:36:58.000511 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:36:58.000511 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:36:58.000511 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:36:58.000511 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:36:58.000511 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:36:58.000511 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:36:58.000511 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:36:58.000511 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:36:58.000511 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:36:58.000511 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:36:58.000511 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:36:58.000511 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:36:58.000511 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:36:58.000511 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:36:58.000511 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:36:58.000511 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:36:58.000511 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:36:58.001661 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:36:58.001661 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:36:58.001661 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:36:58.001661 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 600/1210] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:36:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:36:58.001905 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12940ef-1d0f-45a6-9d95-9ba1d19c54c1 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:36:58.108578 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:36:58.111610 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12940ef-1d0f-45a6-9d95-9ba1d19c54c1 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:36:58.166426 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a791f4-8799-43ed-b563-7782d4c3c798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:36:58.166971 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a791f4-8799-43ed-b563-7782d4c3c798 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:36:58.167249 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a791f4-8799-43ed-b563-7782d4c3c798 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:36:58.167555 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a791f4-8799-43ed-b563-7782d4c3c798 None admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:36:58.168194 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a791f4-8799-43ed-b563-7782d4c3c798 None admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:36:58.168544 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a791f4-8799-43ed-b563-7782d4c3c798 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:36:58.169995 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a791f4-8799-43ed-b563-7782d4c3c798 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:36:58.186774 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 611/1211] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:36:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:36:58.188030 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ee6ed-dc1c-4afb-804c-705d265a7c6a None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:36:58.298746 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ee6ed-dc1c-4afb-804c-705d265a7c6a None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:36:58.302035 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:36:58.353396 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5148f911-0114-42a7-a7bc-ab7c537ebd8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:36:58.354503 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5148f911-0114-42a7-a7bc-ab7c537ebd8f None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:36:58.355060 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5148f911-0114-42a7-a7bc-ab7c537ebd8f None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:36:58.355651 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5148f911-0114-42a7-a7bc-ab7c537ebd8f None admin] PATH_INFO: `/v3/users/placement` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:36:58.364730 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5148f911-0114-42a7-a7bc-ab7c537ebd8f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:36:58.366709 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5148f911-0114-42a7-a7bc-ab7c537ebd8f None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:36:58.381289 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-5148f911-0114-42a7-a7bc-ab7c537ebd8f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 11 10:36:58.381289 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:36:58.381289 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:36:58.381289 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:36:58.381289 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:36:58.381289 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:36:58.381289 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:36:58.381289 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:36:58.381289 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:36:58.381289 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:36:58.381289 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:36:58.381289 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:36:58.381289 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 10:36:58.381289 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 10:36:58.381289 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 10:36:58.381289 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 10:36:58.382517 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:36:58.382517 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:36:58.382517 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 10:36:58.382517 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:36:58.382517 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 10:36:58.382517 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:36:58.382517 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:36:58.382517 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:36:58.382517 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:36:58.382517 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:36:58.382517 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:36:58.382517 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:36:58.382517 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:36:58.382517 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:36:58.382517 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:36:58.382517 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:36:58.382517 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:36:58.382517 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:36:58.384005 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:36:58.384005 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:36:58.384005 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 10:36:58.384005 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 10:36:58.384005 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 10:36:58.384005 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 10:36:58.384005 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 10:36:58.384005 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 10:36:58.384005 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 11 10:36:58.384005 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:36:58.384005 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 601/1212] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:36:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:36:58.388448 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a791f4-8799-43ed-b563-7782d4c3c798 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:36:58.523453 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a791f4-8799-43ed-b563-7782d4c3c798 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:36:58.568495 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4995f623-ea46-4286-9888-6277283d1f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:36:58.569557 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4995f623-ea46-4286-9888-6277283d1f9e None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:36:58.569840 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4995f623-ea46-4286-9888-6277283d1f9e None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:36:58.570154 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4995f623-ea46-4286-9888-6277283d1f9e None admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:36:58.571253 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4995f623-ea46-4286-9888-6277283d1f9e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:36:58.571662 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4995f623-ea46-4286-9888-6277283d1f9e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:36:58.574167 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4995f623-ea46-4286-9888-6277283d1f9e None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:36:58.604660 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 612/1213] 10.222.0.15 () {56 vars in 1143 bytes} [Sun Sep 11 10:36:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:36:58.607291 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5148f911-0114-42a7-a7bc-ab7c537ebd8f None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:36:58.746843 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5148f911-0114-42a7-a7bc-ab7c537ebd8f None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:36:58.791185 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed4524f-266f-4119-9024-7fc73ed62930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:36:58.792012 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed4524f-266f-4119-9024-7fc73ed62930 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:36:58.792507 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed4524f-266f-4119-9024-7fc73ed62930 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:36:58.793000 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed4524f-266f-4119-9024-7fc73ed62930 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:36:58.839400 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:36:58.890827 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed4524f-266f-4119-9024-7fc73ed62930 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:36:58.893437 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed4524f-266f-4119-9024-7fc73ed62930 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:36:59.020052 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 602/1214] 10.222.0.15 () {58 vars in 1282 bytes} [Sun Sep 11 10:36:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 11 10:36:59.027479 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4995f623-ea46-4286-9888-6277283d1f9e None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:36:59.050836 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:36:59.158011 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4995f623-ea46-4286-9888-6277283d1f9e None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:36:59.198841 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b36c53-c5f0-40d5-9ab4-be94e5c43333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:36:59.199482 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b36c53-c5f0-40d5-9ab4-be94e5c43333 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:36:59.199779 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b36c53-c5f0-40d5-9ab4-be94e5c43333 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:36:59.200025 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b36c53-c5f0-40d5-9ab4-be94e5c43333 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:36:59.211247 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b36c53-c5f0-40d5-9ab4-be94e5c43333 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:36:59.212722 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b36c53-c5f0-40d5-9ab4-be94e5c43333 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:36:59.230936 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-65b36c53-c5f0-40d5-9ab4-be94e5c43333 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:36:59.230936 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:36:59.230936 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:36:59.230936 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:36:59.230936 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:36:59.230936 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:36:59.230936 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:36:59.230936 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:36:59.230936 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:36:59.230936 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:36:59.230936 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:36:59.230936 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:36:59.230936 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:36:59.230936 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:36:59.230936 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:36:59.230936 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:36:59.236196 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:36:59.236196 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:36:59.236196 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:36:59.236196 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:36:59.236196 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:36:59.236196 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:36:59.236196 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:36:59.236196 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:36:59.236196 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:36:59.236196 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:36:59.236196 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:36:59.236196 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:36:59.236196 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:36:59.236196 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:36:59.236196 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:36:59.236196 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:36:59.236196 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:36:59.237746 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:36:59.237746 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:36:59.237746 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:36:59.237746 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 613/1215] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:36:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:36:59.238075 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed4524f-266f-4119-9024-7fc73ed62930 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:36:59.305252 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:36:59.354659 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed4524f-266f-4119-9024-7fc73ed62930 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:36:59.390121 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b625cde-185d-4367-b480-4b3545fcadb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:36:59.390733 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b625cde-185d-4367-b480-4b3545fcadb7 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:36:59.391014 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b625cde-185d-4367-b480-4b3545fcadb7 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:36:59.391284 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b625cde-185d-4367-b480-4b3545fcadb7 None admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:36:59.391952 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b625cde-185d-4367-b480-4b3545fcadb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:36:59.392253 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b625cde-185d-4367-b480-4b3545fcadb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:36:59.393644 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b625cde-185d-4367-b480-4b3545fcadb7 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:36:59.405258 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 603/1216] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:36:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:36:59.411899 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b36c53-c5f0-40d5-9ab4-be94e5c43333 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:36:59.503345 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:36:59.551025 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b36c53-c5f0-40d5-9ab4-be94e5c43333 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:36:59.593716 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6beaaa-aa3d-4043-847e-65dd0734076b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:36:59.595002 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6beaaa-aa3d-4043-847e-65dd0734076b None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:36:59.595254 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6beaaa-aa3d-4043-847e-65dd0734076b None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:36:59.595584 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6beaaa-aa3d-4043-847e-65dd0734076b None admin] PATH_INFO: `/v3/projects/service` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:36:59.604369 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6beaaa-aa3d-4043-847e-65dd0734076b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:36:59.606418 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6beaaa-aa3d-4043-847e-65dd0734076b None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:36:59.619878 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-1b6beaaa-aa3d-4043-847e-65dd0734076b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:36:59.619878 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:36:59.619878 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:36:59.619878 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:36:59.619878 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:36:59.619878 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:36:59.619878 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:36:59.619878 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:36:59.619878 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:36:59.619878 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:36:59.619878 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:36:59.619878 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:36:59.619878 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 10:36:59.619878 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 10:36:59.619878 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 10:36:59.620832 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 10:36:59.620832 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:36:59.620832 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:36:59.620832 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:36:59.620832 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:36:59.620832 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:36:59.620832 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:36:59.620832 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:36:59.620832 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:36:59.620832 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:36:59.620832 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:36:59.620832 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:36:59.620832 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:36:59.620832 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:36:59.620832 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:36:59.620832 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:36:59.620832 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:36:59.621745 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 10:36:59.621745 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 10:36:59.621745 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 10:36:59.621745 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 10:36:59.621745 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 10:36:59.621745 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 10:36:59.621745 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:36:59.621745 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:36:59.621745 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 614/1217] 10.222.0.15 () {56 vars in 1111 bytes} [Sun Sep 11 10:36:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:36:59.627490 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b625cde-185d-4367-b480-4b3545fcadb7 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:36:59.744695 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b625cde-185d-4367-b480-4b3545fcadb7 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:36:59.767105 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:36:59.781240 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d1cf36-223a-4a9c-b407-e750c049915d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:36:59.781839 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d1cf36-223a-4a9c-b407-e750c049915d None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:36:59.782120 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d1cf36-223a-4a9c-b407-e750c049915d None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:36:59.782395 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d1cf36-223a-4a9c-b407-e750c049915d None admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:36:59.783085 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d1cf36-223a-4a9c-b407-e750c049915d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:36:59.783391 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d1cf36-223a-4a9c-b407-e750c049915d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:36:59.785537 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d1cf36-223a-4a9c-b407-e750c049915d None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:36:59.797904 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 604/1218] 10.222.0.15 () {56 vars in 1148 bytes} [Sun Sep 11 10:36:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:36:59.804784 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6beaaa-aa3d-4043-847e-65dd0734076b None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:36:59.921282 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6beaaa-aa3d-4043-847e-65dd0734076b None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:36:59.953417 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:36:59.959496 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1494d9d-5a1e-4f8d-ae44-919e2af2a5ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:36:59.960106 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1494d9d-5a1e-4f8d-ae44-919e2af2a5ea None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:36:59.960382 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1494d9d-5a1e-4f8d-ae44-919e2af2a5ea None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:36:59.960650 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1494d9d-5a1e-4f8d-ae44-919e2af2a5ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:36:59.961357 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1494d9d-5a1e-4f8d-ae44-919e2af2a5ea None admin] RBAC: Adding query filter params (role.id=3cb7ebbc707d453586b3d0f4ab3cf103, scope.project.id=5e1f845f99f04202b83621cb65e0adc2, user.id=08477b4c88f04c9e97391aba401879e0) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:36:59.961703 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1494d9d-5a1e-4f8d-ae44-919e2af2a5ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:36:59.963129 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1494d9d-5a1e-4f8d-ae44-919e2af2a5ea None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:36:59.968428 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 615/1219] 10.222.0.15 () {56 vars in 1374 bytes} [Sun Sep 11 10:36:59 2022] GET /identity/v3/role_assignments?user.id=08477b4c88f04c9e97391aba401879e0&scope.project.id=5e1f845f99f04202b83621cb65e0adc2&role.id=3cb7ebbc707d453586b3d0f4ab3cf103 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:37:02.457484 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4823ad4e-3163-46c3-98ad-439ca9e128fa None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:02.458217 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4823ad4e-3163-46c3-98ad-439ca9e128fa None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:02.458217 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4823ad4e-3163-46c3-98ad-439ca9e128fa None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:02.459357 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 605/1220] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:37:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:37:02.465245 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf38d21c-33c7-4ccb-ab12-957ec10e6d5f None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:02.465719 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf38d21c-33c7-4ccb-ab12-957ec10e6d5f None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:02.465820 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf38d21c-33c7-4ccb-ab12-957ec10e6d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:02.487870 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-cf38d21c-33c7-4ccb-ab12-957ec10e6d5f None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:37:02.489308 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-cf38d21c-33c7-4ccb-ab12-957ec10e6d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:37:02.565425 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 616/1221] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:37:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 10:37:02.853577 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4823ad4e-3163-46c3-98ad-439ca9e128fa None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:02.962599 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4823ad4e-3163-46c3-98ad-439ca9e128fa None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:02.999789 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e900ab-d5d1-43bb-b76a-acb4d9c6ce5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:03.000444 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e900ab-d5d1-43bb-b76a-acb4d9c6ce5e None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:03.000710 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e900ab-d5d1-43bb-b76a-acb4d9c6ce5e None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:03.000987 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e900ab-d5d1-43bb-b76a-acb4d9c6ce5e None admin] PATH_INFO: `/v3/roles/service` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:03.012659 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e900ab-d5d1-43bb-b76a-acb4d9c6ce5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:37:03.017426 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-f1e900ab-d5d1-43bb-b76a-acb4d9c6ce5e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 10:37:03.017426 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:37:03.017426 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:37:03.017426 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:37:03.017426 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:37:03.017426 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:37:03.017426 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:37:03.017426 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:37:03.017426 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:37:03.017426 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:37:03.017426 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:37:03.017426 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:37:03.017426 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:37:03.017426 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:37:03.017426 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:37:03.017426 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:37:03.018810 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:37:03.018810 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:37:03.018810 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:37:03.018810 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:37:03.018810 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:37:03.018810 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:37:03.018810 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:37:03.018810 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:37:03.018810 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:37:03.018810 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:37:03.018810 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:37:03.018810 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:37:03.018810 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:37:03.018810 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:37:03.018810 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:37:03.018810 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:37:03.019689 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:37:03.019689 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:37:03.019689 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:37:03.019689 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:37:03.019689 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:37:03.019689 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:37:03.019689 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:37:03.019689 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:37:03.019689 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:37:03.019689 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:37:03.019689 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 10:37:03.019689 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:37:03.019689 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 606/1222] 10.222.0.15 () {56 vars in 1102 bytes} [Sun Sep 11 10:37:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:37:03.025168 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf38d21c-33c7-4ccb-ab12-957ec10e6d5f None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:03.135253 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf38d21c-33c7-4ccb-ab12-957ec10e6d5f None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:03.159008 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:37:03.170678 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b824f115-ea3c-477d-a020-3d345013b315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:03.171362 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b824f115-ea3c-477d-a020-3d345013b315 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:03.171677 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b824f115-ea3c-477d-a020-3d345013b315 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:03.171950 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b824f115-ea3c-477d-a020-3d345013b315 None admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:03.172636 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b824f115-ea3c-477d-a020-3d345013b315 None admin] RBAC: Adding query filter params (name=service) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:37:03.172913 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b824f115-ea3c-477d-a020-3d345013b315 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:37:03.174897 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b824f115-ea3c-477d-a020-3d345013b315 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:37:03.183373 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 617/1223] 10.222.0.15 () {56 vars in 1103 bytes} [Sun Sep 11 10:37:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:37:03.189191 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e900ab-d5d1-43bb-b76a-acb4d9c6ce5e None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:03.235046 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:37:03.297423 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e900ab-d5d1-43bb-b76a-acb4d9c6ce5e None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:03.337333 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4f5d90-c576-4703-9af3-54a26ab84714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:03.337966 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f5d90-c576-4703-9af3-54a26ab84714 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:03.338261 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f5d90-c576-4703-9af3-54a26ab84714 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:03.338570 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f5d90-c576-4703-9af3-54a26ab84714 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:03.350774 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4f5d90-c576-4703-9af3-54a26ab84714 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:37:03.352551 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4f5d90-c576-4703-9af3-54a26ab84714 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:37:03.369728 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-bf4f5d90-c576-4703-9af3-54a26ab84714 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:37:03.369728 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:37:03.369728 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:37:03.369728 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:37:03.369728 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:37:03.369728 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:37:03.369728 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:37:03.369728 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:37:03.369728 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:37:03.369728 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:37:03.369728 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:37:03.369728 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:37:03.369728 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:37:03.369728 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:37:03.369728 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:37:03.369728 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:37:03.370737 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:37:03.370737 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:37:03.370737 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:37:03.370737 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:37:03.370737 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:37:03.370737 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:37:03.370737 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:37:03.370737 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:37:03.370737 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:37:03.370737 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:37:03.370737 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:37:03.370737 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:37:03.370737 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:37:03.370737 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:37:03.370737 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:37:03.370737 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:37:03.370737 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:37:03.371735 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:37:03.371735 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:37:03.371735 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:37:03.371735 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 607/1224] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:37:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:37:03.377626 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b824f115-ea3c-477d-a020-3d345013b315 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:03.492431 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b824f115-ea3c-477d-a020-3d345013b315 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:03.526960 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d62827-0859-4032-9da6-70642ed44945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:03.527603 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d62827-0859-4032-9da6-70642ed44945 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:03.527883 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d62827-0859-4032-9da6-70642ed44945 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:03.528158 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d62827-0859-4032-9da6-70642ed44945 None admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:03.528824 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d62827-0859-4032-9da6-70642ed44945 None admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:37:03.529127 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d62827-0859-4032-9da6-70642ed44945 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:37:03.530519 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d62827-0859-4032-9da6-70642ed44945 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:37:03.544346 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 618/1225] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:37:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:37:03.551094 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4f5d90-c576-4703-9af3-54a26ab84714 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:03.648948 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:37:03.655767 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4f5d90-c576-4703-9af3-54a26ab84714 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:03.692241 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ea05c0-5291-474c-a387-2bcb1bbc02f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:03.692974 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ea05c0-5291-474c-a387-2bcb1bbc02f2 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:03.693264 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ea05c0-5291-474c-a387-2bcb1bbc02f2 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:03.693537 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ea05c0-5291-474c-a387-2bcb1bbc02f2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:03.702982 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ea05c0-5291-474c-a387-2bcb1bbc02f2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:37:03.704796 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ea05c0-5291-474c-a387-2bcb1bbc02f2 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:37:03.719366 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-b1ea05c0-5291-474c-a387-2bcb1bbc02f2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 11 10:37:03.719366 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:37:03.719366 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:37:03.719366 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:37:03.719366 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:37:03.719366 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:37:03.719366 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:37:03.719366 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:37:03.719366 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:37:03.719366 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:37:03.719366 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:37:03.719366 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:37:03.719366 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 10:37:03.719366 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 10:37:03.719366 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 10:37:03.719366 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 10:37:03.720521 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:37:03.720521 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:37:03.720521 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 10:37:03.720521 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:37:03.720521 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 10:37:03.720521 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:37:03.720521 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:37:03.720521 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:37:03.720521 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:37:03.720521 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:37:03.720521 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:37:03.720521 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:37:03.720521 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:37:03.720521 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:37:03.720521 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:37:03.720521 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:37:03.720521 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:37:03.720521 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:37:03.722315 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:37:03.722315 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:37:03.722315 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 10:37:03.722315 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 10:37:03.722315 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 10:37:03.722315 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 10:37:03.722315 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 10:37:03.722315 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 10:37:03.722315 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 11 10:37:03.722315 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:37:03.727451 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 608/1226] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:37:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:37:03.727583 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d62827-0859-4032-9da6-70642ed44945 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:03.777361 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:37:03.838121 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d62827-0859-4032-9da6-70642ed44945 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:03.876707 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafd96f3-a567-4476-bd7c-6715f15470ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:03.878050 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd96f3-a567-4476-bd7c-6715f15470ae None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:03.878356 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd96f3-a567-4476-bd7c-6715f15470ae None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:03.878661 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd96f3-a567-4476-bd7c-6715f15470ae None admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:03.879445 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafd96f3-a567-4476-bd7c-6715f15470ae None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:37:03.879764 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafd96f3-a567-4476-bd7c-6715f15470ae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:37:03.881303 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafd96f3-a567-4476-bd7c-6715f15470ae None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:37:03.901712 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 619/1227] 10.222.0.15 () {56 vars in 1143 bytes} [Sun Sep 11 10:37:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:37:03.907917 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ea05c0-5291-474c-a387-2bcb1bbc02f2 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:04.022240 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ea05c0-5291-474c-a387-2bcb1bbc02f2 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:04.062610 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a45c46-2a8e-45bf-934e-bace3d6ae5a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:04.063166 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a45c46-2a8e-45bf-934e-bace3d6ae5a8 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:04.063477 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a45c46-2a8e-45bf-934e-bace3d6ae5a8 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:04.063756 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a45c46-2a8e-45bf-934e-bace3d6ae5a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:04.075384 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a45c46-2a8e-45bf-934e-bace3d6ae5a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:37:04.077134 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a45c46-2a8e-45bf-934e-bace3d6ae5a8 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:37:04.091752 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-86a45c46-2a8e-45bf-934e-bace3d6ae5a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:37:04.091752 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:37:04.091752 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:37:04.091752 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:37:04.091752 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:37:04.091752 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:37:04.091752 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:37:04.091752 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:37:04.091752 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:37:04.091752 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:37:04.091752 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:37:04.091752 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:37:04.091752 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:37:04.091752 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:37:04.091752 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:37:04.091752 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:37:04.092873 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:37:04.092873 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:37:04.092873 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:37:04.092873 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:37:04.092873 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:37:04.092873 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:37:04.092873 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:37:04.092873 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:37:04.092873 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:37:04.092873 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:37:04.092873 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:37:04.092873 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:37:04.092873 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:37:04.092873 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:37:04.092873 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:37:04.092873 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:37:04.092873 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:37:04.093934 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:37:04.093934 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:37:04.093934 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:37:04.093934 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 609/1228] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:37:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 10:37:04.099105 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafd96f3-a567-4476-bd7c-6715f15470ae None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:04.202868 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafd96f3-a567-4476-bd7c-6715f15470ae None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:04.205785 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:37:04.243489 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d25c0a3-723a-4acf-8f78-55e45255c934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:04.244098 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25c0a3-723a-4acf-8f78-55e45255c934 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:04.244512 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25c0a3-723a-4acf-8f78-55e45255c934 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:04.244890 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25c0a3-723a-4acf-8f78-55e45255c934 None admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:04.245741 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d25c0a3-723a-4acf-8f78-55e45255c934 None admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:37:04.246040 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d25c0a3-723a-4acf-8f78-55e45255c934 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:37:04.247683 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d25c0a3-723a-4acf-8f78-55e45255c934 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:37:04.261973 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 620/1229] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:37:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:37:04.268091 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a45c46-2a8e-45bf-934e-bace3d6ae5a8 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:04.277904 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:37:04.384508 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a45c46-2a8e-45bf-934e-bace3d6ae5a8 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:04.420527 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7436dba-6d3e-46e9-8ffc-dc560cae867f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:04.421162 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7436dba-6d3e-46e9-8ffc-dc560cae867f None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:04.421443 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7436dba-6d3e-46e9-8ffc-dc560cae867f None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:04.421715 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7436dba-6d3e-46e9-8ffc-dc560cae867f None admin] PATH_INFO: `/v3/projects/service` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:04.428401 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7436dba-6d3e-46e9-8ffc-dc560cae867f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:37:04.429836 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7436dba-6d3e-46e9-8ffc-dc560cae867f None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:37:04.441705 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-f7436dba-6d3e-46e9-8ffc-dc560cae867f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:37:04.441705 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:37:04.441705 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:37:04.441705 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:37:04.441705 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:37:04.441705 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:37:04.441705 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:37:04.441705 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:37:04.441705 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:37:04.441705 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:37:04.441705 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:37:04.441705 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:37:04.441705 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 10:37:04.441705 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 10:37:04.441705 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 10:37:04.442925 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 10:37:04.442925 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:37:04.442925 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:37:04.442925 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:37:04.442925 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:37:04.442925 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:37:04.442925 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:37:04.442925 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:37:04.442925 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:37:04.442925 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:37:04.442925 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:37:04.442925 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:37:04.442925 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:37:04.442925 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:37:04.442925 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:37:04.442925 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:37:04.442925 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:37:04.444156 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 10:37:04.444156 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 10:37:04.444156 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 10:37:04.444156 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 10:37:04.444156 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 10:37:04.444156 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 10:37:04.444156 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:37:04.444156 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:37:04.444156 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 610/1230] 10.222.0.15 () {56 vars in 1111 bytes} [Sun Sep 11 10:37:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 10:37:04.449296 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d25c0a3-723a-4acf-8f78-55e45255c934 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:04.554428 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d25c0a3-723a-4acf-8f78-55e45255c934 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:04.592392 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66899e1b-bed4-415a-afda-63db4f09f61b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:04.592990 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66899e1b-bed4-415a-afda-63db4f09f61b None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:04.593268 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66899e1b-bed4-415a-afda-63db4f09f61b None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:04.593539 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66899e1b-bed4-415a-afda-63db4f09f61b None admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:04.594209 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66899e1b-bed4-415a-afda-63db4f09f61b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:37:04.594511 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66899e1b-bed4-415a-afda-63db4f09f61b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:37:04.596126 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66899e1b-bed4-415a-afda-63db4f09f61b None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:37:04.608464 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 621/1231] 10.222.0.15 () {56 vars in 1148 bytes} [Sun Sep 11 10:37:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:37:04.615907 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7436dba-6d3e-46e9-8ffc-dc560cae867f None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:04.692426 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:37:04.733917 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7436dba-6d3e-46e9-8ffc-dc560cae867f None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:04.771668 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d952d788-266a-4b4a-9c40-e48e822fbc9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:04.772375 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d952d788-266a-4b4a-9c40-e48e822fbc9e None admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:04.772764 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d952d788-266a-4b4a-9c40-e48e822fbc9e None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:04.773063 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d952d788-266a-4b4a-9c40-e48e822fbc9e None admin] PATH_INFO: `/v3/projects/5e1f845f99f04202b83621cb65e0adc2/users/08477b4c88f04c9e97391aba401879e0/roles/3cb7ebbc707d453586b3d0f4ab3cf103` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:04.788682 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d952d788-266a-4b4a-9c40-e48e822fbc9e None admin] RBAC: Authorizing `identity:create_grant(project_id=5e1f845f99f04202b83621cb65e0adc2, user_id=08477b4c88f04c9e97391aba401879e0, role_id=3cb7ebbc707d453586b3d0f4ab3cf103)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:37:04.790752 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d952d788-266a-4b4a-9c40-e48e822fbc9e None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:37:04.803697 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 611/1232] 10.222.0.15 () {58 vars in 1439 bytes} [Sun Sep 11 10:37:04 2022] PUT /identity/v3/projects/5e1f845f99f04202b83621cb65e0adc2/users/08477b4c88f04c9e97391aba401879e0/roles/3cb7ebbc707d453586b3d0f4ab3cf103 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:37:07.120027 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97aaf54-1919-496f-9933-2e23f11e4779 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:07.120639 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97aaf54-1919-496f-9933-2e23f11e4779 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:07.120639 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97aaf54-1919-496f-9933-2e23f11e4779 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:07.121929 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 622/1233] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:37:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 10:37:07.128934 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7861a953-4feb-4dff-a991-cf3b33421abe None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:07.129435 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7861a953-4feb-4dff-a991-cf3b33421abe None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:07.129600 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7861a953-4feb-4dff-a991-cf3b33421abe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:07.149335 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-7861a953-4feb-4dff-a991-cf3b33421abe None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:37:07.150517 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-7861a953-4feb-4dff-a991-cf3b33421abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:37:07.223996 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 612/1234] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:37:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 10:37:07.498500 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97aaf54-1919-496f-9933-2e23f11e4779 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:07.525018 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:37:07.610236 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97aaf54-1919-496f-9933-2e23f11e4779 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:07.649056 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17babef-7bb0-4e66-80d5-c88524f798de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:07.649693 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17babef-7bb0-4e66-80d5-c88524f798de None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:07.649975 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17babef-7bb0-4e66-80d5-c88524f798de None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:07.650247 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17babef-7bb0-4e66-80d5-c88524f798de None admin] PATH_INFO: `/v3/roles/service` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:07.662319 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17babef-7bb0-4e66-80d5-c88524f798de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:37:07.667532 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-d17babef-7bb0-4e66-80d5-c88524f798de None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 10:37:07.667532 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:37:07.667532 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:37:07.667532 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:37:07.667532 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:37:07.667532 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:37:07.667532 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:37:07.667532 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:37:07.667532 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:37:07.667532 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:37:07.667532 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:37:07.667532 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:37:07.667532 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:37:07.667532 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:37:07.667532 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:37:07.667532 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:37:07.668716 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:37:07.668716 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:37:07.668716 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:37:07.668716 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:37:07.668716 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:37:07.668716 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:37:07.668716 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:37:07.668716 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:37:07.668716 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:37:07.668716 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:37:07.668716 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:37:07.668716 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:37:07.668716 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:37:07.668716 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:37:07.668716 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:37:07.668716 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:37:07.669601 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:37:07.669601 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:37:07.669601 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:37:07.669601 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:37:07.669601 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:37:07.669601 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:37:07.669601 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:37:07.669601 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:37:07.669601 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:37:07.669601 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:37:07.669601 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 10:37:07.669601 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:37:07.669601 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 623/1235] 10.222.0.15 () {56 vars in 1102 bytes} [Sun Sep 11 10:37:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:37:07.674871 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7861a953-4feb-4dff-a991-cf3b33421abe None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:07.728732 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:37:07.790872 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7861a953-4feb-4dff-a991-cf3b33421abe None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:07.828108 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed3bf16-02c8-4ac2-8039-a853cc7975fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:07.828719 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed3bf16-02c8-4ac2-8039-a853cc7975fd None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:07.829007 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed3bf16-02c8-4ac2-8039-a853cc7975fd None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:07.829274 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed3bf16-02c8-4ac2-8039-a853cc7975fd None admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:07.829933 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed3bf16-02c8-4ac2-8039-a853cc7975fd None admin] RBAC: Adding query filter params (name=service) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:37:07.830311 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed3bf16-02c8-4ac2-8039-a853cc7975fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:37:07.831805 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed3bf16-02c8-4ac2-8039-a853cc7975fd None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:37:07.839864 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 613/1236] 10.222.0.15 () {56 vars in 1103 bytes} [Sun Sep 11 10:37:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:37:07.846972 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17babef-7bb0-4e66-80d5-c88524f798de None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:07.934861 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:37:07.952373 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17babef-7bb0-4e66-80d5-c88524f798de None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:07.992327 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e7ce3-176e-40cf-85ca-991aab233bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:07.992961 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e7ce3-176e-40cf-85ca-991aab233bc6 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:07.993245 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e7ce3-176e-40cf-85ca-991aab233bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:07.993520 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e7ce3-176e-40cf-85ca-991aab233bc6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:08.004129 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0e7ce3-176e-40cf-85ca-991aab233bc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:37:08.005601 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0e7ce3-176e-40cf-85ca-991aab233bc6 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:37:08.020677 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-de0e7ce3-176e-40cf-85ca-991aab233bc6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:37:08.020677 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:37:08.020677 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:37:08.020677 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:37:08.020677 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:37:08.020677 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:37:08.020677 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:37:08.020677 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:37:08.020677 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:37:08.020677 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:37:08.020677 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:37:08.020677 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:37:08.020677 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:37:08.020677 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:37:08.020677 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:37:08.020677 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:37:08.021869 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:37:08.021869 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:37:08.021869 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:37:08.021869 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:37:08.021869 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:37:08.021869 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:37:08.021869 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:37:08.021869 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:37:08.021869 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:37:08.021869 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:37:08.021869 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:37:08.021869 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:37:08.021869 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:37:08.021869 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:37:08.021869 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:37:08.021869 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:37:08.021869 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:37:08.022886 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:37:08.022886 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:37:08.022886 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:37:08.022886 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 624/1237] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:37:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:37:08.027833 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed3bf16-02c8-4ac2-8039-a853cc7975fd None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:08.145712 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed3bf16-02c8-4ac2-8039-a853cc7975fd None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:08.154819 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:37:08.181088 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ac1954-e5cd-417e-975b-b558f8c9ca94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:08.181696 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ac1954-e5cd-417e-975b-b558f8c9ca94 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:08.181977 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ac1954-e5cd-417e-975b-b558f8c9ca94 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:08.182244 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ac1954-e5cd-417e-975b-b558f8c9ca94 None admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:08.182876 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ac1954-e5cd-417e-975b-b558f8c9ca94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:37:08.183179 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ac1954-e5cd-417e-975b-b558f8c9ca94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:37:08.184580 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ac1954-e5cd-417e-975b-b558f8c9ca94 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:37:08.196722 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 614/1238] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:37:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:37:08.202420 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e7ce3-176e-40cf-85ca-991aab233bc6 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:08.316019 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e7ce3-176e-40cf-85ca-991aab233bc6 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:08.352759 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5743d877-48ce-4bc4-9f13-46fdbbfd737e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:08.353664 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5743d877-48ce-4bc4-9f13-46fdbbfd737e None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:08.353948 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5743d877-48ce-4bc4-9f13-46fdbbfd737e None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:08.354212 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5743d877-48ce-4bc4-9f13-46fdbbfd737e None admin] PATH_INFO: `/v3/users/placement` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:08.357005 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:37:08.363121 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5743d877-48ce-4bc4-9f13-46fdbbfd737e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:37:08.364707 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5743d877-48ce-4bc4-9f13-46fdbbfd737e None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:37:08.376726 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-5743d877-48ce-4bc4-9f13-46fdbbfd737e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 11 10:37:08.376726 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:37:08.376726 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:37:08.376726 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:37:08.376726 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:37:08.376726 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:37:08.376726 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:37:08.376726 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:37:08.376726 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:37:08.376726 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:37:08.376726 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:37:08.376726 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:37:08.376726 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 10:37:08.376726 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 10:37:08.376726 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 10:37:08.376726 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 10:37:08.378279 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:37:08.378279 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:37:08.378279 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 10:37:08.378279 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:37:08.378279 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 10:37:08.378279 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:37:08.378279 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:37:08.378279 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:37:08.378279 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:37:08.378279 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:37:08.378279 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:37:08.378279 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:37:08.378279 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:37:08.378279 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:37:08.378279 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:37:08.378279 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:37:08.378279 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:37:08.378279 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:37:08.379736 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:37:08.379736 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:37:08.379736 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 10:37:08.379736 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 10:37:08.379736 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 10:37:08.379736 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 10:37:08.379736 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 10:37:08.379736 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 10:37:08.379736 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 11 10:37:08.379736 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:37:08.379736 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 625/1239] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:37:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 10:37:08.383705 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ac1954-e5cd-417e-975b-b558f8c9ca94 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:08.490586 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ac1954-e5cd-417e-975b-b558f8c9ca94 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:08.526043 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d857e35a-5f00-4e8e-9eb3-858dc34ec43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:08.526736 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d857e35a-5f00-4e8e-9eb3-858dc34ec43d None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:08.527087 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d857e35a-5f00-4e8e-9eb3-858dc34ec43d None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:08.527378 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d857e35a-5f00-4e8e-9eb3-858dc34ec43d None admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:08.528124 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d857e35a-5f00-4e8e-9eb3-858dc34ec43d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:37:08.528425 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d857e35a-5f00-4e8e-9eb3-858dc34ec43d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:37:08.529843 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d857e35a-5f00-4e8e-9eb3-858dc34ec43d None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:37:08.549411 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 615/1240] 10.222.0.15 () {56 vars in 1143 bytes} [Sun Sep 11 10:37:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:37:08.557519 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5743d877-48ce-4bc4-9f13-46fdbbfd737e None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:08.668380 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5743d877-48ce-4bc4-9f13-46fdbbfd737e None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:08.711253 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13b01bf-3d5e-4b6a-b3d6-74e5d0b66d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:08.712091 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b01bf-3d5e-4b6a-b3d6-74e5d0b66d68 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:08.712382 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b01bf-3d5e-4b6a-b3d6-74e5d0b66d68 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:08.712653 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b01bf-3d5e-4b6a-b3d6-74e5d0b66d68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:08.787282 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:37:08.794457 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13b01bf-3d5e-4b6a-b3d6-74e5d0b66d68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:37:08.796194 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13b01bf-3d5e-4b6a-b3d6-74e5d0b66d68 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:37:08.910756 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 626/1241] 10.222.0.15 () {58 vars in 1282 bytes} [Sun Sep 11 10:37:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 10:37:08.916583 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d857e35a-5f00-4e8e-9eb3-858dc34ec43d None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:08.945779 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:37:09.029638 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d857e35a-5f00-4e8e-9eb3-858dc34ec43d None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:09.072113 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7eb50e-0869-4fb2-90ed-65c5dcdd5607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:09.072676 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7eb50e-0869-4fb2-90ed-65c5dcdd5607 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:09.072955 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7eb50e-0869-4fb2-90ed-65c5dcdd5607 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:09.073227 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7eb50e-0869-4fb2-90ed-65c5dcdd5607 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:09.086687 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7eb50e-0869-4fb2-90ed-65c5dcdd5607 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:37:09.088290 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7eb50e-0869-4fb2-90ed-65c5dcdd5607 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:37:09.105318 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-5d7eb50e-0869-4fb2-90ed-65c5dcdd5607 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:37:09.105318 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:37:09.105318 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:37:09.105318 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:37:09.105318 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:37:09.105318 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:37:09.105318 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:37:09.105318 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:37:09.105318 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:37:09.105318 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:37:09.105318 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:37:09.105318 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:37:09.105318 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:37:09.105318 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:37:09.105318 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:37:09.105318 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:37:09.106613 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:37:09.106613 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:37:09.106613 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:37:09.106613 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:37:09.106613 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:37:09.106613 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:37:09.106613 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:37:09.106613 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:37:09.106613 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:37:09.106613 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:37:09.106613 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:37:09.106613 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:37:09.106613 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:37:09.106613 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:37:09.106613 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:37:09.106613 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:37:09.106613 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:37:09.107562 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:37:09.107562 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:37:09.107562 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:37:09.107562 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 616/1242] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:37:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:37:09.113683 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13b01bf-3d5e-4b6a-b3d6-74e5d0b66d68 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:09.221995 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:37:09.229969 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13b01bf-3d5e-4b6a-b3d6-74e5d0b66d68 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:09.270010 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75d67d4-6dbc-4296-9628-e73eab5eb285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:09.270647 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75d67d4-6dbc-4296-9628-e73eab5eb285 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:09.271749 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75d67d4-6dbc-4296-9628-e73eab5eb285 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:09.272070 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75d67d4-6dbc-4296-9628-e73eab5eb285 None admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:09.272875 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75d67d4-6dbc-4296-9628-e73eab5eb285 None admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:37:09.273216 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75d67d4-6dbc-4296-9628-e73eab5eb285 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:37:09.274793 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75d67d4-6dbc-4296-9628-e73eab5eb285 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:37:09.289999 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 627/1243] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:37:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:37:09.296060 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7eb50e-0869-4fb2-90ed-65c5dcdd5607 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:09.383054 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:37:09.406746 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7eb50e-0869-4fb2-90ed-65c5dcdd5607 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:09.444296 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7edf52-56cb-404f-abfd-ecf5b7a03549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:09.444934 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7edf52-56cb-404f-abfd-ecf5b7a03549 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:09.445221 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7edf52-56cb-404f-abfd-ecf5b7a03549 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:09.445486 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7edf52-56cb-404f-abfd-ecf5b7a03549 None admin] PATH_INFO: `/v3/projects/service` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:09.452269 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7edf52-56cb-404f-abfd-ecf5b7a03549 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:37:09.453706 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7edf52-56cb-404f-abfd-ecf5b7a03549 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:37:09.467205 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-6d7edf52-56cb-404f-abfd-ecf5b7a03549 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:37:09.467205 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:37:09.467205 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:37:09.467205 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:37:09.467205 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:37:09.467205 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:37:09.467205 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:37:09.467205 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:37:09.467205 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:37:09.467205 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:37:09.467205 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:37:09.467205 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:37:09.467205 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 10:37:09.467205 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 10:37:09.467205 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 10:37:09.469414 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 10:37:09.469414 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:37:09.469414 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:37:09.469414 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:37:09.469414 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:37:09.469414 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:37:09.469414 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:37:09.469414 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:37:09.469414 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:37:09.469414 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:37:09.469414 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:37:09.469414 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:37:09.469414 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:37:09.469414 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:37:09.469414 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:37:09.469414 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:37:09.469414 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:37:09.470364 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 10:37:09.470364 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 10:37:09.470364 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 10:37:09.470364 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 10:37:09.470364 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 10:37:09.470364 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 10:37:09.470364 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:37:09.470364 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:37:09.470364 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 617/1244] 10.222.0.15 () {56 vars in 1111 bytes} [Sun Sep 11 10:37:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:37:09.474916 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75d67d4-6dbc-4296-9628-e73eab5eb285 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:09.591992 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75d67d4-6dbc-4296-9628-e73eab5eb285 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:09.631132 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304c8c56-43e2-4801-9377-c9cfc9cea8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:09.631988 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304c8c56-43e2-4801-9377-c9cfc9cea8e7 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:09.632501 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304c8c56-43e2-4801-9377-c9cfc9cea8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:09.632776 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304c8c56-43e2-4801-9377-c9cfc9cea8e7 None admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:09.633692 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304c8c56-43e2-4801-9377-c9cfc9cea8e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:37:09.634096 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304c8c56-43e2-4801-9377-c9cfc9cea8e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:37:09.635852 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304c8c56-43e2-4801-9377-c9cfc9cea8e7 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:37:09.651687 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 628/1245] 10.222.0.15 () {56 vars in 1148 bytes} [Sun Sep 11 10:37:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:37:09.659952 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7edf52-56cb-404f-abfd-ecf5b7a03549 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:09.778787 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7edf52-56cb-404f-abfd-ecf5b7a03549 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:09.816143 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908b3ea6-db5b-4a4c-859e-a5474ef313f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:09.816847 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908b3ea6-db5b-4a4c-859e-a5474ef313f7 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:09.817204 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908b3ea6-db5b-4a4c-859e-a5474ef313f7 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:09.817519 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908b3ea6-db5b-4a4c-859e-a5474ef313f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:09.818254 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908b3ea6-db5b-4a4c-859e-a5474ef313f7 None admin] RBAC: Adding query filter params (role.id=3cb7ebbc707d453586b3d0f4ab3cf103, scope.project.id=5e1f845f99f04202b83621cb65e0adc2, user.id=08477b4c88f04c9e97391aba401879e0) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:37:09.818561 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908b3ea6-db5b-4a4c-859e-a5474ef313f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:37:09.820029 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908b3ea6-db5b-4a4c-859e-a5474ef313f7 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:37:09.828142 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 618/1246] 10.222.0.15 () {56 vars in 1374 bytes} [Sun Sep 11 10:37:09 2022] GET /identity/v3/role_assignments?user.id=08477b4c88f04c9e97391aba401879e0&scope.project.id=5e1f845f99f04202b83621cb65e0adc2&role.id=3cb7ebbc707d453586b3d0f4ab3cf103 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:37:12.127793 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ea6dee-543a-49d2-a4e1-15abbb47a957 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:12.128335 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ea6dee-543a-49d2-a4e1-15abbb47a957 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:12.128335 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ea6dee-543a-49d2-a4e1-15abbb47a957 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:12.129693 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 629/1247] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:37:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:37:12.135863 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54654b6-bd60-4527-b71e-50ffd627e7e3 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:12.136370 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54654b6-bd60-4527-b71e-50ffd627e7e3 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:12.136460 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54654b6-bd60-4527-b71e-50ffd627e7e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:12.143367 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:37:12.157124 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-e54654b6-bd60-4527-b71e-50ffd627e7e3 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:37:12.158989 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-e54654b6-bd60-4527-b71e-50ffd627e7e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:37:12.230780 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 619/1248] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:37:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 10:37:12.505374 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ea6dee-543a-49d2-a4e1-15abbb47a957 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:12.524925 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:37:12.607558 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ea6dee-543a-49d2-a4e1-15abbb47a957 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:12.640004 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bedf28e-917b-439f-9ae0-1de0dd208f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:12.640622 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bedf28e-917b-439f-9ae0-1de0dd208f24 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:12.640902 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bedf28e-917b-439f-9ae0-1de0dd208f24 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:12.641174 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bedf28e-917b-439f-9ae0-1de0dd208f24 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:12.651577 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bedf28e-917b-439f-9ae0-1de0dd208f24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:37:12.656246 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-4bedf28e-917b-439f-9ae0-1de0dd208f24 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 10:37:12.656246 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:37:12.656246 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:37:12.656246 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:37:12.656246 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:37:12.656246 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:37:12.656246 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:37:12.656246 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:37:12.656246 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:37:12.656246 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:37:12.656246 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:37:12.656246 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:37:12.656246 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:37:12.656246 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:37:12.656246 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:37:12.656246 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:37:12.657322 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:37:12.657322 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:37:12.657322 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:37:12.657322 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:37:12.657322 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:37:12.657322 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:37:12.657322 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:37:12.657322 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:37:12.657322 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:37:12.657322 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:37:12.657322 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:37:12.657322 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:37:12.657322 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:37:12.657322 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:37:12.657322 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:37:12.657322 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:37:12.658379 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:37:12.658379 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:37:12.658379 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:37:12.658379 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:37:12.658379 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:37:12.658379 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:37:12.658379 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:37:12.658379 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:37:12.658379 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:37:12.658379 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:37:12.658379 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 10:37:12.658379 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:37:12.658379 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 630/1249] 10.222.0.15 () {56 vars in 1096 bytes} [Sun Sep 11 10:37:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 10:37:12.663459 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54654b6-bd60-4527-b71e-50ffd627e7e3 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:12.768258 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54654b6-bd60-4527-b71e-50ffd627e7e3 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:12.796628 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:37:12.801809 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cf88bb-7cce-4be7-9bb1-9cf76008033f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:12.802410 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf88bb-7cce-4be7-9bb1-9cf76008033f None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:12.802686 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf88bb-7cce-4be7-9bb1-9cf76008033f None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:12.802953 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf88bb-7cce-4be7-9bb1-9cf76008033f None admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:12.803630 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cf88bb-7cce-4be7-9bb1-9cf76008033f None admin] RBAC: Adding query filter params (name=admin) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:37:12.803931 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cf88bb-7cce-4be7-9bb1-9cf76008033f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:37:12.805278 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cf88bb-7cce-4be7-9bb1-9cf76008033f None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:37:12.813016 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 620/1250] 10.222.0.15 () {56 vars in 1099 bytes} [Sun Sep 11 10:37:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:37:12.818841 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bedf28e-917b-439f-9ae0-1de0dd208f24 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:12.894418 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:37:12.917316 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bedf28e-917b-439f-9ae0-1de0dd208f24 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:12.949317 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10757853-8bad-4d68-a12b-872786f7b88b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:12.949933 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10757853-8bad-4d68-a12b-872786f7b88b None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:12.950217 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10757853-8bad-4d68-a12b-872786f7b88b None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:12.950489 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10757853-8bad-4d68-a12b-872786f7b88b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:12.960740 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10757853-8bad-4d68-a12b-872786f7b88b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:37:12.962191 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10757853-8bad-4d68-a12b-872786f7b88b None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:37:12.976998 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-10757853-8bad-4d68-a12b-872786f7b88b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:37:12.976998 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:37:12.976998 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:37:12.976998 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:37:12.976998 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:37:12.976998 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:37:12.976998 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:37:12.976998 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:37:12.976998 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:37:12.976998 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:37:12.976998 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:37:12.976998 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:37:12.976998 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:37:12.976998 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:37:12.976998 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:37:12.976998 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:37:12.978430 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:37:12.978430 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:37:12.978430 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:37:12.978430 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:37:12.978430 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:37:12.978430 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:37:12.978430 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:37:12.978430 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:37:12.978430 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:37:12.978430 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:37:12.978430 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:37:12.978430 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:37:12.978430 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:37:12.978430 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:37:12.978430 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:37:12.978430 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:37:12.978430 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:37:12.979727 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:37:12.979727 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:37:12.979727 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:37:12.979727 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 631/1251] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:37:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:37:12.984077 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cf88bb-7cce-4be7-9bb1-9cf76008033f None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:13.081655 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cf88bb-7cce-4be7-9bb1-9cf76008033f None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:13.114367 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3cce86-9309-42be-a3c0-b3230b3409a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:13.114962 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3cce86-9309-42be-a3c0-b3230b3409a0 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:13.115239 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3cce86-9309-42be-a3c0-b3230b3409a0 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:13.115544 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3cce86-9309-42be-a3c0-b3230b3409a0 None admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:13.116162 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3cce86-9309-42be-a3c0-b3230b3409a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:37:13.116460 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3cce86-9309-42be-a3c0-b3230b3409a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:37:13.117798 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3cce86-9309-42be-a3c0-b3230b3409a0 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:37:13.130066 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 621/1252] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:37:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:37:13.135691 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10757853-8bad-4d68-a12b-872786f7b88b None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:13.232756 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10757853-8bad-4d68-a12b-872786f7b88b None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:13.257851 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:37:13.266054 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fd4157-85cd-446c-a217-8f4b04dd7d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:13.266672 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fd4157-85cd-446c-a217-8f4b04dd7d5d None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:13.266950 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fd4157-85cd-446c-a217-8f4b04dd7d5d None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:13.267219 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fd4157-85cd-446c-a217-8f4b04dd7d5d None admin] PATH_INFO: `/v3/users/placement` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:13.274951 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fd4157-85cd-446c-a217-8f4b04dd7d5d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:37:13.276412 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fd4157-85cd-446c-a217-8f4b04dd7d5d None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:37:13.287337 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-c4fd4157-85cd-446c-a217-8f4b04dd7d5d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 11 10:37:13.287337 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:37:13.287337 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:37:13.287337 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:37:13.287337 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:37:13.287337 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:37:13.287337 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:37:13.287337 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:37:13.287337 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:37:13.287337 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:37:13.287337 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:37:13.287337 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:37:13.287337 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 10:37:13.287337 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 10:37:13.287337 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 10:37:13.287337 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 10:37:13.288511 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:37:13.288511 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:37:13.288511 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 10:37:13.288511 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:37:13.288511 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 10:37:13.288511 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:37:13.288511 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:37:13.288511 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:37:13.288511 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:37:13.288511 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:37:13.288511 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:37:13.288511 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:37:13.288511 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:37:13.288511 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:37:13.288511 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:37:13.288511 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:37:13.288511 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:37:13.288511 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:37:13.289510 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:37:13.289510 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:37:13.289510 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 10:37:13.289510 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 10:37:13.289510 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 10:37:13.289510 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 10:37:13.289510 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 10:37:13.289510 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 10:37:13.289510 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 11 10:37:13.289510 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:37:13.289510 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 632/1253] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:37:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:37:13.294152 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3cce86-9309-42be-a3c0-b3230b3409a0 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:13.349480 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:37:13.394401 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3cce86-9309-42be-a3c0-b3230b3409a0 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:13.426933 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f628dae-ce69-4753-b99c-50f4bcc9c531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:13.427547 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f628dae-ce69-4753-b99c-50f4bcc9c531 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:13.427825 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f628dae-ce69-4753-b99c-50f4bcc9c531 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:13.428094 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f628dae-ce69-4753-b99c-50f4bcc9c531 None admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:13.428802 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f628dae-ce69-4753-b99c-50f4bcc9c531 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:37:13.429102 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f628dae-ce69-4753-b99c-50f4bcc9c531 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:37:13.430492 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f628dae-ce69-4753-b99c-50f4bcc9c531 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:37:13.448012 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 622/1254] 10.222.0.15 () {56 vars in 1143 bytes} [Sun Sep 11 10:37:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:37:13.453910 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fd4157-85cd-446c-a217-8f4b04dd7d5d None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:13.550940 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fd4157-85cd-446c-a217-8f4b04dd7d5d None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:13.583883 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14a49c6-9332-49f3-b735-574dd93af507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:13.584476 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a49c6-9332-49f3-b735-574dd93af507 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:13.584757 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a49c6-9332-49f3-b735-574dd93af507 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:13.585031 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a49c6-9332-49f3-b735-574dd93af507 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:13.638819 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:37:13.651471 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14a49c6-9332-49f3-b735-574dd93af507 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:37:13.652923 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14a49c6-9332-49f3-b735-574dd93af507 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:37:13.756567 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 633/1255] 10.222.0.15 () {58 vars in 1282 bytes} [Sun Sep 11 10:37:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 10:37:13.762411 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f628dae-ce69-4753-b99c-50f4bcc9c531 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:13.859284 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f628dae-ce69-4753-b99c-50f4bcc9c531 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:13.872681 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:37:13.893253 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b702db-83ef-473f-9e97-28da59423444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:13.893803 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b702db-83ef-473f-9e97-28da59423444 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:13.894081 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b702db-83ef-473f-9e97-28da59423444 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:13.894352 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b702db-83ef-473f-9e97-28da59423444 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:13.903954 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b702db-83ef-473f-9e97-28da59423444 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:37:13.905375 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b702db-83ef-473f-9e97-28da59423444 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:37:13.919953 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-47b702db-83ef-473f-9e97-28da59423444 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:37:13.919953 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:37:13.919953 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:37:13.919953 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:37:13.919953 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:37:13.919953 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:37:13.919953 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:37:13.919953 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:37:13.919953 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:37:13.919953 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:37:13.919953 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:37:13.919953 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:37:13.919953 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:37:13.919953 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:37:13.919953 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:37:13.919953 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:37:13.921074 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:37:13.921074 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:37:13.921074 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:37:13.921074 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:37:13.921074 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:37:13.921074 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:37:13.921074 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:37:13.921074 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:37:13.921074 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:37:13.921074 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:37:13.921074 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:37:13.921074 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:37:13.921074 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:37:13.921074 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:37:13.921074 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:37:13.921074 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:37:13.921074 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:37:13.921998 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:37:13.921998 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:37:13.921998 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:37:13.921998 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 623/1256] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:37:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:37:13.927350 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14a49c6-9332-49f3-b735-574dd93af507 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:14.013372 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:37:14.025471 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14a49c6-9332-49f3-b735-574dd93af507 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:14.058731 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e3bac0-57c7-4977-af25-1124c29846e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:14.059327 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e3bac0-57c7-4977-af25-1124c29846e2 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:14.059646 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e3bac0-57c7-4977-af25-1124c29846e2 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:14.059921 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e3bac0-57c7-4977-af25-1124c29846e2 None admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:14.060549 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e3bac0-57c7-4977-af25-1124c29846e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:37:14.060850 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e3bac0-57c7-4977-af25-1124c29846e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:37:14.062201 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e3bac0-57c7-4977-af25-1124c29846e2 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:37:14.073512 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 634/1257] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:37:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:37:14.079158 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b702db-83ef-473f-9e97-28da59423444 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:14.175969 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b702db-83ef-473f-9e97-28da59423444 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:14.208133 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4a2e45-3ed3-4d78-b5ba-91c51e1000e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:14.208740 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a2e45-3ed3-4d78-b5ba-91c51e1000e9 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:14.209017 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a2e45-3ed3-4d78-b5ba-91c51e1000e9 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:14.209286 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a2e45-3ed3-4d78-b5ba-91c51e1000e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:14.215692 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4a2e45-3ed3-4d78-b5ba-91c51e1000e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:37:14.217137 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4a2e45-3ed3-4d78-b5ba-91c51e1000e9 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:37:14.228108 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-9b4a2e45-3ed3-4d78-b5ba-91c51e1000e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:37:14.228108 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:37:14.228108 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:37:14.228108 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:37:14.228108 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:37:14.228108 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:37:14.228108 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:37:14.228108 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:37:14.228108 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:37:14.228108 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:37:14.228108 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:37:14.228108 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:37:14.228108 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 10:37:14.228108 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 10:37:14.228108 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 10:37:14.229367 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 10:37:14.229367 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:37:14.229367 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:37:14.229367 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:37:14.229367 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:37:14.229367 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:37:14.229367 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:37:14.229367 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:37:14.229367 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:37:14.229367 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:37:14.229367 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:37:14.229367 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:37:14.229367 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:37:14.229367 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:37:14.229367 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:37:14.229367 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:37:14.229367 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:37:14.230656 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 10:37:14.230656 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 10:37:14.230656 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 10:37:14.230656 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 10:37:14.230656 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 10:37:14.230656 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 10:37:14.230656 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:37:14.230656 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:37:14.230656 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 624/1258] 10.222.0.15 () {56 vars in 1111 bytes} [Sun Sep 11 10:37:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:37:14.235189 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e3bac0-57c7-4977-af25-1124c29846e2 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:14.332418 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e3bac0-57c7-4977-af25-1124c29846e2 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:14.364826 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc903b79-ce92-4ee5-b342-1336b3517d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:14.365420 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc903b79-ce92-4ee5-b342-1336b3517d77 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:14.365695 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc903b79-ce92-4ee5-b342-1336b3517d77 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:14.365963 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc903b79-ce92-4ee5-b342-1336b3517d77 None admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:14.366633 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc903b79-ce92-4ee5-b342-1336b3517d77 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:37:14.366930 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc903b79-ce92-4ee5-b342-1336b3517d77 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:37:14.368362 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc903b79-ce92-4ee5-b342-1336b3517d77 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:37:14.380353 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 635/1259] 10.222.0.15 () {56 vars in 1148 bytes} [Sun Sep 11 10:37:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:37:14.386416 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4a2e45-3ed3-4d78-b5ba-91c51e1000e9 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:14.411621 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:37:14.487986 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4a2e45-3ed3-4d78-b5ba-91c51e1000e9 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:14.522051 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e774320e-14a2-4b0b-adf8-80f2c9c7b635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:14.522646 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e774320e-14a2-4b0b-adf8-80f2c9c7b635 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:14.522928 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e774320e-14a2-4b0b-adf8-80f2c9c7b635 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:14.523203 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e774320e-14a2-4b0b-adf8-80f2c9c7b635 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:14.523956 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e774320e-14a2-4b0b-adf8-80f2c9c7b635 None admin] RBAC: Adding query filter params (role.id=2cb7d73f0ea8414296367573f6902340, scope.project.id=5e1f845f99f04202b83621cb65e0adc2, user.id=08477b4c88f04c9e97391aba401879e0) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:37:14.524265 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e774320e-14a2-4b0b-adf8-80f2c9c7b635 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:37:14.525679 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e774320e-14a2-4b0b-adf8-80f2c9c7b635 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:37:14.530898 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 625/1260] 10.222.0.15 () {56 vars in 1374 bytes} [Sun Sep 11 10:37:14 2022] GET /identity/v3/role_assignments?user.id=08477b4c88f04c9e97391aba401879e0&scope.project.id=5e1f845f99f04202b83621cb65e0adc2&role.id=2cb7d73f0ea8414296367573f6902340 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:37:16.663488 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f9382-d7d5-45ba-be16-93008c1f9db0 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:16.664045 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f9382-d7d5-45ba-be16-93008c1f9db0 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:16.664045 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f9382-d7d5-45ba-be16-93008c1f9db0 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:16.665317 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 636/1261] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:37:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:37:16.670594 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7066c7ec-d424-457f-ba7d-beaeffdf2a91 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:16.671017 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7066c7ec-d424-457f-ba7d-beaeffdf2a91 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:16.671141 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7066c7ec-d424-457f-ba7d-beaeffdf2a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:16.690476 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-7066c7ec-d424-457f-ba7d-beaeffdf2a91 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:37:16.691801 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-7066c7ec-d424-457f-ba7d-beaeffdf2a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:37:16.761650 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 626/1262] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:37:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 10:37:17.030122 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6f9382-d7d5-45ba-be16-93008c1f9db0 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:17.041326 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:37:17.129798 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6f9382-d7d5-45ba-be16-93008c1f9db0 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:17.162298 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53c4234-0582-4c63-a348-6bb8a3ee8318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:17.162909 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c4234-0582-4c63-a348-6bb8a3ee8318 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:17.163187 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c4234-0582-4c63-a348-6bb8a3ee8318 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:17.163478 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c4234-0582-4c63-a348-6bb8a3ee8318 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:17.173043 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53c4234-0582-4c63-a348-6bb8a3ee8318 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:37:17.178274 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-c53c4234-0582-4c63-a348-6bb8a3ee8318 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 10:37:17.178274 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:37:17.178274 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:37:17.178274 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:37:17.178274 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:37:17.178274 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:37:17.178274 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:37:17.178274 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:37:17.178274 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:37:17.178274 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:37:17.178274 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:37:17.178274 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:37:17.178274 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:37:17.178274 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:37:17.178274 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:37:17.178274 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:37:17.179452 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:37:17.179452 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:37:17.179452 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:37:17.179452 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:37:17.179452 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:37:17.179452 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:37:17.179452 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:37:17.179452 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:37:17.179452 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:37:17.179452 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:37:17.179452 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:37:17.179452 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:37:17.179452 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:37:17.179452 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:37:17.179452 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:37:17.179452 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:37:17.180761 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:37:17.180761 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:37:17.180761 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:37:17.180761 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:37:17.180761 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:37:17.180761 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:37:17.180761 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:37:17.180761 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:37:17.180761 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:37:17.180761 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:37:17.180761 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 10:37:17.180761 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:37:17.180761 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 637/1263] 10.222.0.15 () {56 vars in 1096 bytes} [Sun Sep 11 10:37:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:37:17.185281 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7066c7ec-d424-457f-ba7d-beaeffdf2a91 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:17.203873 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:37:17.281277 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7066c7ec-d424-457f-ba7d-beaeffdf2a91 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:17.315388 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbc8eff-787e-4732-8fa7-59ce980cdd00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:17.316021 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbc8eff-787e-4732-8fa7-59ce980cdd00 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:17.316299 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbc8eff-787e-4732-8fa7-59ce980cdd00 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:17.316568 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbc8eff-787e-4732-8fa7-59ce980cdd00 None admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:17.317217 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbc8eff-787e-4732-8fa7-59ce980cdd00 None admin] RBAC: Adding query filter params (name=admin) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:37:17.317516 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbc8eff-787e-4732-8fa7-59ce980cdd00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:37:17.318872 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbc8eff-787e-4732-8fa7-59ce980cdd00 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:37:17.326637 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 627/1264] 10.222.0.15 () {56 vars in 1099 bytes} [Sun Sep 11 10:37:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:37:17.332795 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53c4234-0582-4c63-a348-6bb8a3ee8318 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:17.397371 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:37:17.430001 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53c4234-0582-4c63-a348-6bb8a3ee8318 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:17.463661 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7084c6-d99a-4574-bccc-391a52630c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:17.464282 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7084c6-d99a-4574-bccc-391a52630c59 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:17.464563 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7084c6-d99a-4574-bccc-391a52630c59 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:17.464835 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7084c6-d99a-4574-bccc-391a52630c59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:17.474566 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7084c6-d99a-4574-bccc-391a52630c59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:37:17.476048 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7084c6-d99a-4574-bccc-391a52630c59 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:37:17.492017 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-cb7084c6-d99a-4574-bccc-391a52630c59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:37:17.492017 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:37:17.492017 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:37:17.492017 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:37:17.492017 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:37:17.492017 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:37:17.492017 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:37:17.492017 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:37:17.492017 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:37:17.492017 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:37:17.492017 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:37:17.492017 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:37:17.492017 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:37:17.492017 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:37:17.492017 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:37:17.492017 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:37:17.497551 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:37:17.497551 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:37:17.497551 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:37:17.497551 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:37:17.497551 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:37:17.497551 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:37:17.497551 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:37:17.497551 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:37:17.497551 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:37:17.497551 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:37:17.497551 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:37:17.497551 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:37:17.497551 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:37:17.497551 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:37:17.497551 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:37:17.497551 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:37:17.497551 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:37:17.498709 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:37:17.498709 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:37:17.498709 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:37:17.498709 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 638/1265] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:37:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:37:17.499013 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbc8eff-787e-4732-8fa7-59ce980cdd00 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:17.577473 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:37:17.596501 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbc8eff-787e-4732-8fa7-59ce980cdd00 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:17.629028 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1c4b75-60db-4703-af32-430dadd94b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:17.629611 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c4b75-60db-4703-af32-430dadd94b26 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:17.629888 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c4b75-60db-4703-af32-430dadd94b26 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:17.630154 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c4b75-60db-4703-af32-430dadd94b26 None admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:17.630775 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1c4b75-60db-4703-af32-430dadd94b26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:37:17.631070 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1c4b75-60db-4703-af32-430dadd94b26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:37:17.632438 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1c4b75-60db-4703-af32-430dadd94b26 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:37:17.643551 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 628/1266] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:37:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:37:17.649150 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7084c6-d99a-4574-bccc-391a52630c59 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:17.750265 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7084c6-d99a-4574-bccc-391a52630c59 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:17.770022 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:37:17.783770 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f638b8-cb5e-4cef-b247-2b8b8c72bbc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:17.784390 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f638b8-cb5e-4cef-b247-2b8b8c72bbc7 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:17.784667 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f638b8-cb5e-4cef-b247-2b8b8c72bbc7 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:17.784936 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f638b8-cb5e-4cef-b247-2b8b8c72bbc7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:17.792007 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f638b8-cb5e-4cef-b247-2b8b8c72bbc7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:37:17.793428 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f638b8-cb5e-4cef-b247-2b8b8c72bbc7 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:37:17.804168 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-98f638b8-cb5e-4cef-b247-2b8b8c72bbc7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 11 10:37:17.804168 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:37:17.804168 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:37:17.804168 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:37:17.804168 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:37:17.804168 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:37:17.804168 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:37:17.804168 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:37:17.804168 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:37:17.804168 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:37:17.804168 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:37:17.804168 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:37:17.804168 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 10:37:17.804168 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 10:37:17.804168 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 10:37:17.804168 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 10:37:17.805303 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:37:17.805303 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:37:17.805303 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 10:37:17.805303 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:37:17.805303 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 10:37:17.805303 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:37:17.805303 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:37:17.805303 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:37:17.805303 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:37:17.805303 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:37:17.805303 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:37:17.805303 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:37:17.805303 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:37:17.805303 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:37:17.805303 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:37:17.805303 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:37:17.805303 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:37:17.805303 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:37:17.806444 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:37:17.806444 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:37:17.806444 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 10:37:17.806444 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 10:37:17.806444 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 10:37:17.806444 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 10:37:17.806444 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 10:37:17.806444 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 10:37:17.806444 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 11 10:37:17.806444 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:37:17.806444 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 639/1267] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:37:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:37:17.811041 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1c4b75-60db-4703-af32-430dadd94b26 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:17.906952 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1c4b75-60db-4703-af32-430dadd94b26 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:17.940159 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e87ac1c-93b5-4b7b-b180-fd08914b18c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:17.940750 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e87ac1c-93b5-4b7b-b180-fd08914b18c7 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:17.941027 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e87ac1c-93b5-4b7b-b180-fd08914b18c7 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:17.941301 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e87ac1c-93b5-4b7b-b180-fd08914b18c7 None admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:17.942008 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e87ac1c-93b5-4b7b-b180-fd08914b18c7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:37:17.942307 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e87ac1c-93b5-4b7b-b180-fd08914b18c7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:37:17.943738 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e87ac1c-93b5-4b7b-b180-fd08914b18c7 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:37:17.952019 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:37:17.960378 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 629/1268] 10.222.0.15 () {56 vars in 1143 bytes} [Sun Sep 11 10:37:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:37:17.966092 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f638b8-cb5e-4cef-b247-2b8b8c72bbc7 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:18.064571 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f638b8-cb5e-4cef-b247-2b8b8c72bbc7 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:18.097162 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1914c4-7276-43c4-98ab-6628afb0ee7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:18.097782 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1914c4-7276-43c4-98ab-6628afb0ee7d None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:18.098062 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1914c4-7276-43c4-98ab-6628afb0ee7d None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:18.098332 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1914c4-7276-43c4-98ab-6628afb0ee7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:18.108742 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1914c4-7276-43c4-98ab-6628afb0ee7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:37:18.110172 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1914c4-7276-43c4-98ab-6628afb0ee7d None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:37:18.123669 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-ad1914c4-7276-43c4-98ab-6628afb0ee7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:37:18.123669 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:37:18.123669 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:37:18.123669 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:37:18.123669 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:37:18.123669 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:37:18.123669 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:37:18.123669 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:37:18.123669 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:37:18.123669 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:37:18.123669 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:37:18.123669 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:37:18.123669 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:37:18.123669 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:37:18.123669 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:37:18.123669 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:37:18.124763 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:37:18.124763 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:37:18.124763 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:37:18.124763 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:37:18.124763 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:37:18.124763 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:37:18.124763 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:37:18.124763 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:37:18.124763 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:37:18.124763 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:37:18.124763 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:37:18.124763 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:37:18.124763 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:37:18.124763 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:37:18.124763 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:37:18.124763 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:37:18.124763 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:37:18.125754 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:37:18.125754 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:37:18.125754 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:37:18.125754 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 640/1269] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:37:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:37:18.130494 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e87ac1c-93b5-4b7b-b180-fd08914b18c7 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:18.227855 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e87ac1c-93b5-4b7b-b180-fd08914b18c7 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:18.260469 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625c7104-120d-4ff8-84ff-74d64b09ba25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:18.261055 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625c7104-120d-4ff8-84ff-74d64b09ba25 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:18.261332 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625c7104-120d-4ff8-84ff-74d64b09ba25 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:18.261599 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625c7104-120d-4ff8-84ff-74d64b09ba25 None admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:18.262219 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625c7104-120d-4ff8-84ff-74d64b09ba25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:37:18.262516 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625c7104-120d-4ff8-84ff-74d64b09ba25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:37:18.263884 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625c7104-120d-4ff8-84ff-74d64b09ba25 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:37:18.275276 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 630/1270] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:37:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:37:18.280882 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1914c4-7276-43c4-98ab-6628afb0ee7d None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:18.312780 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:37:18.382641 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1914c4-7276-43c4-98ab-6628afb0ee7d None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:18.418027 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60a6c86-1167-4682-a085-d39bcd709281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:18.418644 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a6c86-1167-4682-a085-d39bcd709281 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:18.418924 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a6c86-1167-4682-a085-d39bcd709281 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:18.419199 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a6c86-1167-4682-a085-d39bcd709281 None admin] PATH_INFO: `/v3/projects/service` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:18.425877 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60a6c86-1167-4682-a085-d39bcd709281 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:37:18.427310 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60a6c86-1167-4682-a085-d39bcd709281 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:37:18.439827 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-b60a6c86-1167-4682-a085-d39bcd709281 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:37:18.439827 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:37:18.439827 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:37:18.439827 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:37:18.439827 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:37:18.439827 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:37:18.439827 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:37:18.439827 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:37:18.439827 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:37:18.439827 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:37:18.439827 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:37:18.439827 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:37:18.439827 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 10:37:18.439827 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 10:37:18.439827 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 10:37:18.441128 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 10:37:18.441128 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:37:18.441128 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:37:18.441128 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:37:18.441128 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:37:18.441128 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:37:18.441128 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:37:18.441128 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:37:18.441128 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:37:18.441128 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:37:18.441128 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:37:18.441128 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:37:18.441128 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:37:18.441128 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:37:18.441128 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:37:18.441128 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:37:18.441128 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:37:18.442061 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 10:37:18.442061 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 10:37:18.442061 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 10:37:18.442061 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 10:37:18.442061 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 10:37:18.442061 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 10:37:18.442061 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:37:18.442061 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:37:18.442061 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 641/1271] 10.222.0.15 () {56 vars in 1111 bytes} [Sun Sep 11 10:37:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:37:18.446824 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625c7104-120d-4ff8-84ff-74d64b09ba25 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:18.504779 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:37:18.545692 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625c7104-120d-4ff8-84ff-74d64b09ba25 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:18.578475 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb62dec9-08cb-44c3-96aa-cbe75bd202f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:18.579072 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb62dec9-08cb-44c3-96aa-cbe75bd202f4 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:18.579355 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb62dec9-08cb-44c3-96aa-cbe75bd202f4 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:18.579661 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb62dec9-08cb-44c3-96aa-cbe75bd202f4 None admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:18.581139 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb62dec9-08cb-44c3-96aa-cbe75bd202f4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:37:18.581448 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb62dec9-08cb-44c3-96aa-cbe75bd202f4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:37:18.582863 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb62dec9-08cb-44c3-96aa-cbe75bd202f4 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:37:18.594310 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 631/1272] 10.222.0.15 () {56 vars in 1148 bytes} [Sun Sep 11 10:37:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:37:18.600305 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60a6c86-1167-4682-a085-d39bcd709281 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:18.693954 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:37:18.703827 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60a6c86-1167-4682-a085-d39bcd709281 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:18.737169 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78056a98-20cb-4e11-a85f-59aa59b69159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:18.737823 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78056a98-20cb-4e11-a85f-59aa59b69159 None admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:18.738108 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78056a98-20cb-4e11-a85f-59aa59b69159 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:18.738382 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78056a98-20cb-4e11-a85f-59aa59b69159 None admin] PATH_INFO: `/v3/projects/5e1f845f99f04202b83621cb65e0adc2/users/08477b4c88f04c9e97391aba401879e0/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:18.740276 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78056a98-20cb-4e11-a85f-59aa59b69159 None admin] RBAC: Authorizing `identity:create_grant(project_id=5e1f845f99f04202b83621cb65e0adc2, user_id=08477b4c88f04c9e97391aba401879e0, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:37:18.741701 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78056a98-20cb-4e11-a85f-59aa59b69159 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:37:18.754637 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 642/1273] 10.222.0.15 () {58 vars in 1439 bytes} [Sun Sep 11 10:37:18 2022] PUT /identity/v3/projects/5e1f845f99f04202b83621cb65e0adc2/users/08477b4c88f04c9e97391aba401879e0/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:37:20.898439 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa4c54b-8ae3-45bb-8ef7-703d94622e68 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:20.899037 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa4c54b-8ae3-45bb-8ef7-703d94622e68 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:20.899037 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa4c54b-8ae3-45bb-8ef7-703d94622e68 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:20.900314 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 632/1274] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:37:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:37:20.905510 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d0c7e2-43a9-4787-a4a5-161e3420fbfe None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:20.905947 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d0c7e2-43a9-4787-a4a5-161e3420fbfe None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:20.906046 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d0c7e2-43a9-4787-a4a5-161e3420fbfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:20.924300 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-12d0c7e2-43a9-4787-a4a5-161e3420fbfe None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:37:20.925477 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-12d0c7e2-43a9-4787-a4a5-161e3420fbfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:37:20.989209 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 643/1275] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:37:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 10:37:21.255206 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa4c54b-8ae3-45bb-8ef7-703d94622e68 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:21.355001 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa4c54b-8ae3-45bb-8ef7-703d94622e68 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:21.372315 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:37:21.389339 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f158cb68-f238-404c-9da7-bfecc9e93803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:21.389959 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f158cb68-f238-404c-9da7-bfecc9e93803 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:21.390236 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f158cb68-f238-404c-9da7-bfecc9e93803 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:21.390505 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f158cb68-f238-404c-9da7-bfecc9e93803 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:21.400810 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f158cb68-f238-404c-9da7-bfecc9e93803 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 10:37:21.405479 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-f158cb68-f238-404c-9da7-bfecc9e93803 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 10:37:21.405479 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:37:21.405479 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:37:21.405479 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:37:21.405479 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:37:21.405479 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:37:21.405479 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:37:21.405479 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:37:21.405479 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:37:21.405479 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:37:21.405479 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:37:21.405479 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:37:21.405479 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 10:37:21.405479 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 10:37:21.405479 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 10:37:21.405479 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 10:37:21.406574 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 10:37:21.406574 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 10:37:21.406574 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 10:37:21.406574 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 10:37:21.406574 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:37:21.406574 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:37:21.406574 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:37:21.406574 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:37:21.406574 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:37:21.406574 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:37:21.406574 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:37:21.406574 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:37:21.406574 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:37:21.406574 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:37:21.406574 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:37:21.406574 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:37:21.407831 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:37:21.407831 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:37:21.407831 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:37:21.407831 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:37:21.407831 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 10:37:21.407831 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 10:37:21.407831 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 10:37:21.407831 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 10:37:21.407831 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 10:37:21.407831 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 10:37:21.407831 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 10:37:21.407831 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:37:21.407831 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 633/1276] 10.222.0.15 () {56 vars in 1096 bytes} [Sun Sep 11 10:37:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:37:21.412519 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d0c7e2-43a9-4787-a4a5-161e3420fbfe None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:21.490222 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:37:21.510312 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d0c7e2-43a9-4787-a4a5-161e3420fbfe None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:21.542960 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a285540-83a7-494c-8365-80c689a73db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:21.543582 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a285540-83a7-494c-8365-80c689a73db6 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:21.543864 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a285540-83a7-494c-8365-80c689a73db6 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:21.544132 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a285540-83a7-494c-8365-80c689a73db6 None admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:21.544779 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a285540-83a7-494c-8365-80c689a73db6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:37:21.545077 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a285540-83a7-494c-8365-80c689a73db6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:37:21.546430 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a285540-83a7-494c-8365-80c689a73db6 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:37:21.554181 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 644/1277] 10.222.0.15 () {56 vars in 1099 bytes} [Sun Sep 11 10:37:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:37:21.559995 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f158cb68-f238-404c-9da7-bfecc9e93803 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:21.656901 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f158cb68-f238-404c-9da7-bfecc9e93803 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:21.691988 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41aba4b-c9f3-4fc2-a212-5ddff35eebc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:21.692707 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41aba4b-c9f3-4fc2-a212-5ddff35eebc3 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:21.693069 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41aba4b-c9f3-4fc2-a212-5ddff35eebc3 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:21.693379 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41aba4b-c9f3-4fc2-a212-5ddff35eebc3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:21.706058 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41aba4b-c9f3-4fc2-a212-5ddff35eebc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:37:21.707682 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41aba4b-c9f3-4fc2-a212-5ddff35eebc3 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:37:21.722736 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-d41aba4b-c9f3-4fc2-a212-5ddff35eebc3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:37:21.722736 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:37:21.722736 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:37:21.722736 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:37:21.722736 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:37:21.722736 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:37:21.722736 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:37:21.722736 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:37:21.722736 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:37:21.722736 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:37:21.722736 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:37:21.722736 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:37:21.722736 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:37:21.722736 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:37:21.722736 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:37:21.722736 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:37:21.723975 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:37:21.723975 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:37:21.723975 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:37:21.723975 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:37:21.723975 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:37:21.723975 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:37:21.723975 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:37:21.723975 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:37:21.723975 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:37:21.723975 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:37:21.723975 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:37:21.723975 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:37:21.723975 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:37:21.723975 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:37:21.723975 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:37:21.723975 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:37:21.723975 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:37:21.724894 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:37:21.724894 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:37:21.724894 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:37:21.724894 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 634/1278] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:37:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:37:21.730129 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a285540-83a7-494c-8365-80c689a73db6 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:21.827660 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a285540-83a7-494c-8365-80c689a73db6 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:21.859509 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9d57b9-0e36-457c-a632-f966cdec1e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:21.860105 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d57b9-0e36-457c-a632-f966cdec1e64 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:21.860383 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d57b9-0e36-457c-a632-f966cdec1e64 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:21.860654 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d57b9-0e36-457c-a632-f966cdec1e64 None admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:21.861914 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9d57b9-0e36-457c-a632-f966cdec1e64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:37:21.862212 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9d57b9-0e36-457c-a632-f966cdec1e64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:37:21.863600 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9d57b9-0e36-457c-a632-f966cdec1e64 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:37:21.874750 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 645/1279] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:37:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:37:21.881356 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41aba4b-c9f3-4fc2-a212-5ddff35eebc3 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:21.911168 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:37:21.979663 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41aba4b-c9f3-4fc2-a212-5ddff35eebc3 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:22.013144 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afbf6fa-5b9e-4c85-87ff-7226305f1367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:22.013762 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afbf6fa-5b9e-4c85-87ff-7226305f1367 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:22.014037 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afbf6fa-5b9e-4c85-87ff-7226305f1367 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:22.014303 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afbf6fa-5b9e-4c85-87ff-7226305f1367 None admin] PATH_INFO: `/v3/users/placement` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:22.021564 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afbf6fa-5b9e-4c85-87ff-7226305f1367 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:37:22.022980 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afbf6fa-5b9e-4c85-87ff-7226305f1367 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:37:22.034432 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-0afbf6fa-5b9e-4c85-87ff-7226305f1367 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 11 10:37:22.034432 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:37:22.034432 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:37:22.034432 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:37:22.034432 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:37:22.034432 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:37:22.034432 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:37:22.034432 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:37:22.034432 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:37:22.034432 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:37:22.034432 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:37:22.034432 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:37:22.034432 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 10:37:22.034432 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 10:37:22.034432 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 10:37:22.034432 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 10:37:22.035526 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:37:22.035526 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:37:22.035526 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 10:37:22.035526 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:37:22.035526 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 10:37:22.035526 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 10:37:22.035526 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:37:22.035526 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:37:22.035526 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:37:22.035526 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:37:22.035526 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:37:22.035526 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:37:22.035526 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:37:22.035526 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:37:22.035526 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:37:22.035526 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:37:22.035526 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:37:22.035526 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:37:22.036695 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:37:22.036695 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:37:22.036695 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 10:37:22.036695 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 10:37:22.036695 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 10:37:22.036695 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 10:37:22.036695 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 10:37:22.036695 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 10:37:22.036695 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 11 10:37:22.036695 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:37:22.036695 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 635/1280] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:37:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:37:22.041441 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9d57b9-0e36-457c-a632-f966cdec1e64 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:22.045153 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:37:22.137525 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9d57b9-0e36-457c-a632-f966cdec1e64 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:22.170041 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdcb89b-c403-4c01-a8cc-1430ab1541e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:22.170631 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdcb89b-c403-4c01-a8cc-1430ab1541e0 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:22.170906 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdcb89b-c403-4c01-a8cc-1430ab1541e0 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:22.171170 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdcb89b-c403-4c01-a8cc-1430ab1541e0 None admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:22.171915 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdcb89b-c403-4c01-a8cc-1430ab1541e0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:37:22.172214 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdcb89b-c403-4c01-a8cc-1430ab1541e0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:37:22.173708 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdcb89b-c403-4c01-a8cc-1430ab1541e0 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:37:22.190366 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 646/1281] 10.222.0.15 () {56 vars in 1143 bytes} [Sun Sep 11 10:37:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:37:22.196073 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afbf6fa-5b9e-4c85-87ff-7226305f1367 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:22.284551 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:37:22.294187 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afbf6fa-5b9e-4c85-87ff-7226305f1367 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:22.326757 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288f3cee-f946-483f-8e0d-c9b3808c46af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:22.327357 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f3cee-f946-483f-8e0d-c9b3808c46af None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:22.327664 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f3cee-f946-483f-8e0d-c9b3808c46af None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:22.327936 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f3cee-f946-483f-8e0d-c9b3808c46af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:22.393144 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288f3cee-f946-483f-8e0d-c9b3808c46af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:37:22.394507 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288f3cee-f946-483f-8e0d-c9b3808c46af None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:37:22.479291 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:37:22.490135 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 636/1282] 10.222.0.15 () {58 vars in 1282 bytes} [Sun Sep 11 10:37:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 10:37:22.495958 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdcb89b-c403-4c01-a8cc-1430ab1541e0 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:22.555353 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:37:22.592821 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdcb89b-c403-4c01-a8cc-1430ab1541e0 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:22.626401 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfab0cd5-7401-4318-934c-ff625ea22274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:22.627832 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfab0cd5-7401-4318-934c-ff625ea22274 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:22.628115 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfab0cd5-7401-4318-934c-ff625ea22274 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:22.628385 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfab0cd5-7401-4318-934c-ff625ea22274 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:22.637832 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfab0cd5-7401-4318-934c-ff625ea22274 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:37:22.639265 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfab0cd5-7401-4318-934c-ff625ea22274 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:37:22.653536 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-dfab0cd5-7401-4318-934c-ff625ea22274 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:37:22.653536 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:37:22.653536 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:37:22.653536 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:37:22.653536 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:37:22.653536 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:37:22.653536 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:37:22.653536 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:37:22.653536 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:37:22.653536 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:37:22.653536 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:37:22.653536 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:37:22.653536 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 10:37:22.653536 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 10:37:22.653536 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 10:37:22.653536 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 10:37:22.654622 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:37:22.654622 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:37:22.654622 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:37:22.654622 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:37:22.654622 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:37:22.654622 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:37:22.654622 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:37:22.654622 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:37:22.654622 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:37:22.654622 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:37:22.654622 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:37:22.654622 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:37:22.654622 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:37:22.654622 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:37:22.654622 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:37:22.654622 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:37:22.654622 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 10:37:22.655945 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 10:37:22.655945 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 10:37:22.655945 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:37:22.655945 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 647/1283] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:37:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:37:22.660487 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288f3cee-f946-483f-8e0d-c9b3808c46af None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:22.758577 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288f3cee-f946-483f-8e0d-c9b3808c46af None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:22.791239 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2fce44-be6b-4f3e-99db-326c53fd62c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:22.791853 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2fce44-be6b-4f3e-99db-326c53fd62c8 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:22.792130 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2fce44-be6b-4f3e-99db-326c53fd62c8 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:22.792398 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2fce44-be6b-4f3e-99db-326c53fd62c8 None admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:22.793023 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2fce44-be6b-4f3e-99db-326c53fd62c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:37:22.793320 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2fce44-be6b-4f3e-99db-326c53fd62c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:37:22.794662 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2fce44-be6b-4f3e-99db-326c53fd62c8 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:37:22.805862 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 637/1284] 10.222.0.15 () {56 vars in 1109 bytes} [Sun Sep 11 10:37:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:37:22.811552 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfab0cd5-7401-4318-934c-ff625ea22274 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:22.908713 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfab0cd5-7401-4318-934c-ff625ea22274 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:22.921553 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:37:22.942283 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7ab38-b093-4cb5-bbc4-945434fd0fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:22.942896 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7ab38-b093-4cb5-bbc4-945434fd0fc1 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:22.943172 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7ab38-b093-4cb5-bbc4-945434fd0fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:22.943459 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7ab38-b093-4cb5-bbc4-945434fd0fc1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:22.949821 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e7ab38-b093-4cb5-bbc4-945434fd0fc1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:37:22.951227 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e7ab38-b093-4cb5-bbc4-945434fd0fc1 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:37:22.962347 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application [None req-80e7ab38-b093-4cb5-bbc4-945434fd0fc1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:37:22.962347 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:37:22.962347 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:37:22.962347 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:37:22.962347 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:37:22.962347 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:37:22.962347 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:37:22.962347 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:37:22.962347 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:37:22.962347 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:37:22.962347 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:37:22.962347 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:37:22.962347 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 10:37:22.962347 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 10:37:22.962347 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 10:37:22.963414 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 10:37:22.963414 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:37:22.963414 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:37:22.963414 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:37:22.963414 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:37:22.963414 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:37:22.963414 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:37:22.963414 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:37:22.963414 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application with Lock( Sep 11 10:37:22.963414 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:37:22.963414 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:37:22.963414 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:37:22.963414 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:37:22.963414 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:37:22.963414 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:37:22.963414 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:37:22.963414 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:37:22.964548 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 10:37:22.964548 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 10:37:22.964548 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 10:37:22.964548 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 10:37:22.964548 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 10:37:22.964548 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 10:37:22.964548 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 10:37:22.964548 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: ERROR keystone.server.flask.application Sep 11 10:37:22.964548 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 648/1285] 10.222.0.15 () {56 vars in 1111 bytes} [Sun Sep 11 10:37:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:37:22.969316 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2fce44-be6b-4f3e-99db-326c53fd62c8 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:23.019248 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:37:23.067945 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2fce44-be6b-4f3e-99db-326c53fd62c8 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:23.102580 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0570f1b0-8558-4773-ac80-688d0e1bb041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:23.103177 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0570f1b0-8558-4773-ac80-688d0e1bb041 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:23.103736 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0570f1b0-8558-4773-ac80-688d0e1bb041 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:23.104004 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0570f1b0-8558-4773-ac80-688d0e1bb041 None admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:23.104715 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0570f1b0-8558-4773-ac80-688d0e1bb041 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:37:23.105018 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0570f1b0-8558-4773-ac80-688d0e1bb041 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:37:23.106429 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0570f1b0-8558-4773-ac80-688d0e1bb041 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:37:23.118709 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 638/1286] 10.222.0.15 () {56 vars in 1148 bytes} [Sun Sep 11 10:37:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:37:23.124940 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7ab38-b093-4cb5-bbc4-945434fd0fc1 None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:23.222734 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7ab38-b093-4cb5-bbc4-945434fd0fc1 None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:23.254635 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d53e33-7e5b-4c98-a36c-0861e0c4e817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:23.255216 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d53e33-7e5b-4c98-a36c-0861e0c4e817 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:23.255519 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d53e33-7e5b-4c98-a36c-0861e0c4e817 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:23.255790 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d53e33-7e5b-4c98-a36c-0861e0c4e817 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:23.256499 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d53e33-7e5b-4c98-a36c-0861e0c4e817 None admin] RBAC: Adding query filter params (role.id=2cb7d73f0ea8414296367573f6902340, scope.project.id=5e1f845f99f04202b83621cb65e0adc2, user.id=08477b4c88f04c9e97391aba401879e0) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:37:23.256797 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d53e33-7e5b-4c98-a36c-0861e0c4e817 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:37:23.258195 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d53e33-7e5b-4c98-a36c-0861e0c4e817 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:37:23.263640 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 649/1287] 10.222.0.15 () {56 vars in 1374 bytes} [Sun Sep 11 10:37:23 2022] GET /identity/v3/role_assignments?user.id=08477b4c88f04c9e97391aba401879e0&scope.project.id=5e1f845f99f04202b83621cb65e0adc2&role.id=2cb7d73f0ea8414296367573f6902340 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:37:25.428740 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b6d058-d97f-4e32-aae6-642dabd0256c None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:25.429261 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b6d058-d97f-4e32-aae6-642dabd0256c None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:25.429261 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b6d058-d97f-4e32-aae6-642dabd0256c None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:25.430600 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 639/1288] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:37:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:37:25.435755 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e65732c-f4c7-4811-913e-6a20b9f3ee69 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:25.436126 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e65732c-f4c7-4811-913e-6a20b9f3ee69 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:25.436292 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e65732c-f4c7-4811-913e-6a20b9f3ee69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:25.454111 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-1e65732c-f4c7-4811-913e-6a20b9f3ee69 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:37:25.456111 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-1e65732c-f4c7-4811-913e-6a20b9f3ee69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:37:25.484712 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:37:25.521556 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 650/1289] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:37:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 10:37:25.816252 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b6d058-d97f-4e32-aae6-642dabd0256c None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:25.915640 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b6d058-d97f-4e32-aae6-642dabd0256c None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:25.925938 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:37:25.949083 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f984edb-34db-47ee-ab3f-9914b9592348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:25.949704 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f984edb-34db-47ee-ab3f-9914b9592348 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:25.949982 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f984edb-34db-47ee-ab3f-9914b9592348 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:25.950253 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f984edb-34db-47ee-ab3f-9914b9592348 None admin] PATH_INFO: `/v3/services/placement` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:25.950822 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f984edb-34db-47ee-ab3f-9914b9592348 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:37:25.952210 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f984edb-34db-47ee-ab3f-9914b9592348 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:37:25.961279 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-5f984edb-34db-47ee-ab3f-9914b9592348 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 11 10:37:25.961279 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:37:25.961279 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 10:37:25.961279 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 10:37:25.961279 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 10:37:25.961279 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 10:37:25.961279 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 10:37:25.961279 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 10:37:25.961279 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 11 10:37:25.961279 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:37:25.961279 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 10:37:25.961279 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:37:25.961279 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:37:25.961279 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:37:25.961279 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:37:25.961279 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:37:25.961279 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:37:25.962462 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:37:25.962462 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:37:25.962462 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:37:25.962462 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:37:25.962462 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:37:25.962462 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:37:25.962462 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 10:37:25.962462 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 10:37:25.962462 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 10:37:25.962462 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 10:37:25.962462 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:37:25.962462 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:37:25.962462 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:37:25.962462 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:37:25.962462 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:37:25.962462 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:37:25.963553 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:37:25.963553 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:37:25.963553 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:37:25.963553 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:37:25.963553 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:37:25.963553 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:37:25.963553 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:37:25.963553 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:37:25.963553 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:37:25.963553 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:37:25.963553 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 10:37:25.963553 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 10:37:25.963553 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 11 10:37:25.963553 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:37:25.963553 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 640/1290] 10.222.0.15 () {56 vars in 1117 bytes} [Sun Sep 11 10:37:25 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:37:25.968451 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e65732c-f4c7-4811-913e-6a20b9f3ee69 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:26.062888 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e65732c-f4c7-4811-913e-6a20b9f3ee69 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:26.095808 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac1906a-78d0-4468-b400-8b810d20916e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:26.096337 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac1906a-78d0-4468-b400-8b810d20916e None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:26.096616 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac1906a-78d0-4468-b400-8b810d20916e None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:26.096883 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac1906a-78d0-4468-b400-8b810d20916e None admin] PATH_INFO: `/v3/services` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:26.097495 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac1906a-78d0-4468-b400-8b810d20916e None admin] RBAC: Adding query filter params (name=placement) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:37:26.097793 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac1906a-78d0-4468-b400-8b810d20916e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:37:26.099137 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac1906a-78d0-4468-b400-8b810d20916e None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:37:26.104082 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 651/1291] 10.222.0.15 () {56 vars in 1116 bytes} [Sun Sep 11 10:37:25 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:37:26.109905 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f984edb-34db-47ee-ab3f-9914b9592348 None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:26.207126 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f984edb-34db-47ee-ab3f-9914b9592348 None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:26.239509 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d832f73-e847-4e9d-b4a3-132c1241508b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:26.240097 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d832f73-e847-4e9d-b4a3-132c1241508b None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:26.240375 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d832f73-e847-4e9d-b4a3-132c1241508b None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:26.240647 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d832f73-e847-4e9d-b4a3-132c1241508b None admin] PATH_INFO: `/v3/services` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:26.241262 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d832f73-e847-4e9d-b4a3-132c1241508b None admin] RBAC: Adding query filter params (type=placement) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:37:26.241562 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d832f73-e847-4e9d-b4a3-132c1241508b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:37:26.242911 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d832f73-e847-4e9d-b4a3-132c1241508b None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:37:26.247116 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 641/1292] 10.222.0.15 () {56 vars in 1116 bytes} [Sun Sep 11 10:37:26 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:37:28.381658 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e4cdd1-31c1-458c-9d39-ba81aeccfba2 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:28.382197 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e4cdd1-31c1-458c-9d39-ba81aeccfba2 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:28.382197 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e4cdd1-31c1-458c-9d39-ba81aeccfba2 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:28.383536 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 652/1293] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:37:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:37:28.388609 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b955f58d-4701-4663-9b1a-a51fe80eda31 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:28.388992 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b955f58d-4701-4663-9b1a-a51fe80eda31 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:28.389196 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b955f58d-4701-4663-9b1a-a51fe80eda31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:28.407643 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-b955f58d-4701-4663-9b1a-a51fe80eda31 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:37:28.408910 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-b955f58d-4701-4663-9b1a-a51fe80eda31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:37:28.437730 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:37:28.474396 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 642/1294] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:37:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 10:37:28.752760 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e4cdd1-31c1-458c-9d39-ba81aeccfba2 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:28.783697 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:37:28.852227 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e4cdd1-31c1-458c-9d39-ba81aeccfba2 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:28.885771 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e05623-d5e9-4384-b30e-92bc9d599a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:28.886370 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e05623-d5e9-4384-b30e-92bc9d599a3a None admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:28.886652 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e05623-d5e9-4384-b30e-92bc9d599a3a None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:28.886926 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e05623-d5e9-4384-b30e-92bc9d599a3a None admin] PATH_INFO: `/v3/services` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:28.887588 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e05623-d5e9-4384-b30e-92bc9d599a3a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:37:28.888955 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e05623-d5e9-4384-b30e-92bc9d599a3a None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:37:28.902868 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 653/1295] 10.222.0.15 () {60 vars in 1141 bytes} [Sun Sep 11 10:37:28 2022] POST /identity/v3/services => generated 246 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 10:37:31.047868 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bd1539-fd19-432e-bf98-c1e7f80070ac None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:31.048560 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bd1539-fd19-432e-bf98-c1e7f80070ac None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:31.048667 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bd1539-fd19-432e-bf98-c1e7f80070ac None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:31.049986 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 643/1296] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:37:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:37:31.055269 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b395325-da6e-447f-b535-fe314ce339a9 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:31.055697 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b395325-da6e-447f-b535-fe314ce339a9 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:31.055889 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b395325-da6e-447f-b535-fe314ce339a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:31.074072 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-3b395325-da6e-447f-b535-fe314ce339a9 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:37:31.075249 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-3b395325-da6e-447f-b535-fe314ce339a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:37:31.147442 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 654/1297] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:37:31 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 10:37:31.417002 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bd1539-fd19-432e-bf98-c1e7f80070ac None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:31.515219 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bd1539-fd19-432e-bf98-c1e7f80070ac None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:31.548754 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fd341a-f209-4c11-a582-fb35879b8a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:31.549367 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fd341a-f209-4c11-a582-fb35879b8a4c None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:31.549641 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fd341a-f209-4c11-a582-fb35879b8a4c None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:31.549903 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fd341a-f209-4c11-a582-fb35879b8a4c None admin] PATH_INFO: `/v3/services/placement` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:31.550464 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fd341a-f209-4c11-a582-fb35879b8a4c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:37:31.551845 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fd341a-f209-4c11-a582-fb35879b8a4c None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:37:31.560254 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-f8fd341a-f209-4c11-a582-fb35879b8a4c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 11 10:37:31.560254 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:37:31.560254 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 10:37:31.560254 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 10:37:31.560254 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 10:37:31.560254 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 10:37:31.560254 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 10:37:31.560254 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 10:37:31.560254 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 11 10:37:31.560254 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:37:31.560254 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 10:37:31.560254 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:37:31.560254 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:37:31.560254 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:37:31.560254 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:37:31.560254 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:37:31.560254 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:37:31.561231 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:37:31.561231 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:37:31.561231 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:37:31.561231 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:37:31.561231 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:37:31.561231 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:37:31.561231 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 10:37:31.561231 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 10:37:31.561231 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 10:37:31.561231 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 10:37:31.561231 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:37:31.561231 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:37:31.561231 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:37:31.561231 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:37:31.561231 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:37:31.561231 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:37:31.562415 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:37:31.562415 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:37:31.562415 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:37:31.562415 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:37:31.562415 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:37:31.562415 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:37:31.562415 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:37:31.562415 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:37:31.562415 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:37:31.562415 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:37:31.562415 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 10:37:31.562415 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 10:37:31.562415 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 11 10:37:31.562415 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:37:31.562415 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 644/1298] 10.222.0.15 () {56 vars in 1117 bytes} [Sun Sep 11 10:37:31 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:37:31.567677 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b395325-da6e-447f-b535-fe314ce339a9 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:31.585950 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:37:31.664583 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b395325-da6e-447f-b535-fe314ce339a9 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:31.699940 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1047a4-af22-4009-8c3b-564c40ff3f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:31.700671 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1047a4-af22-4009-8c3b-564c40ff3f2c None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:31.701031 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1047a4-af22-4009-8c3b-564c40ff3f2c None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:31.701347 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1047a4-af22-4009-8c3b-564c40ff3f2c None admin] PATH_INFO: `/v3/services` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:31.701998 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1047a4-af22-4009-8c3b-564c40ff3f2c None admin] RBAC: Adding query filter params (name=placement) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:37:31.702360 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1047a4-af22-4009-8c3b-564c40ff3f2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:37:31.703964 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1047a4-af22-4009-8c3b-564c40ff3f2c None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:37:31.710899 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 655/1299] 10.222.0.15 () {56 vars in 1116 bytes} [Sun Sep 11 10:37:31 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:37:31.717756 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fd341a-f209-4c11-a582-fb35879b8a4c None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:31.751978 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:37:31.819396 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fd341a-f209-4c11-a582-fb35879b8a4c None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:31.852685 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281e8c10-de8f-4ea2-9a28-e53f4782dd62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:31.853527 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e8c10-de8f-4ea2-9a28-e53f4782dd62 None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:31.853816 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e8c10-de8f-4ea2-9a28-e53f4782dd62 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:31.854087 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e8c10-de8f-4ea2-9a28-e53f4782dd62 None admin] PATH_INFO: `/v3/endpoints` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:31.854771 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281e8c10-de8f-4ea2-9a28-e53f4782dd62 None admin] RBAC: Adding query filter params (interface=public, service_id=2f34a21710974b35b5852041929ac298, region_id=RegionOne) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:37:31.855078 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281e8c10-de8f-4ea2-9a28-e53f4782dd62 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:37:31.856466 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281e8c10-de8f-4ea2-9a28-e53f4782dd62 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:37:31.861308 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 645/1300] 10.222.0.15 () {56 vars in 1251 bytes} [Sun Sep 11 10:37:31 2022] GET /identity/v3/endpoints?service_id=2f34a21710974b35b5852041929ac298&interface=public®ion_id=RegionOne => generated 194 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:37:31.866985 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1047a4-af22-4009-8c3b-564c40ff3f2c None admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:31.945908 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:37:31.963601 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1047a4-af22-4009-8c3b-564c40ff3f2c None admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:31.996924 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84872ef7-5fca-4a93-9ae8-fa8b0edf4586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:31.997523 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84872ef7-5fca-4a93-9ae8-fa8b0edf4586 None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:31.997804 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84872ef7-5fca-4a93-9ae8-fa8b0edf4586 None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:31.998077 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84872ef7-5fca-4a93-9ae8-fa8b0edf4586 None admin] PATH_INFO: `/v3/services` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:31.998669 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84872ef7-5fca-4a93-9ae8-fa8b0edf4586 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:37:32.000050 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84872ef7-5fca-4a93-9ae8-fa8b0edf4586 None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:37:32.006986 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 656/1301] 10.222.0.15 () {56 vars in 1087 bytes} [Sun Sep 11 10:37:31 2022] GET /identity/v3/services => generated 2191 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:37:34.133578 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153caca5-ae22-4bcb-82f6-917c96a0d9aa None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:34.134143 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153caca5-ae22-4bcb-82f6-917c96a0d9aa None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:34.134143 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153caca5-ae22-4bcb-82f6-917c96a0d9aa None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:34.135445 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 646/1302] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:37:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:37:34.140621 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b760662-0d73-4f53-a489-024fbed57f89 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:34.141023 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b760662-0d73-4f53-a489-024fbed57f89 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:34.141170 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b760662-0d73-4f53-a489-024fbed57f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:34.159487 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-1b760662-0d73-4f53-a489-024fbed57f89 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:37:34.160657 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-1b760662-0d73-4f53-a489-024fbed57f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:37:34.224448 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 657/1303] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:37:34 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 10:37:34.494627 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153caca5-ae22-4bcb-82f6-917c96a0d9aa None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:34.588221 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:37:34.595004 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153caca5-ae22-4bcb-82f6-917c96a0d9aa None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:34.628603 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9a7452-4909-40f6-819d-a4d5fbc76aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:34.629222 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a7452-4909-40f6-819d-a4d5fbc76aab None admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:34.629505 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a7452-4909-40f6-819d-a4d5fbc76aab None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:34.629778 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a7452-4909-40f6-819d-a4d5fbc76aab None admin] PATH_INFO: `/v3/services/placement` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:34.630351 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9a7452-4909-40f6-819d-a4d5fbc76aab None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:37:34.631738 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9a7452-4909-40f6-819d-a4d5fbc76aab None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:37:34.640215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application [None req-9a9a7452-4909-40f6-819d-a4d5fbc76aab None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 11 10:37:34.640215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:37:34.640215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 10:37:34.640215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 10:37:34.640215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 10:37:34.640215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 10:37:34.640215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 10:37:34.640215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 10:37:34.640215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 11 10:37:34.640215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:37:34.640215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 10:37:34.640215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:37:34.640215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 10:37:34.640215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 10:37:34.640215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 10:37:34.640215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 10:37:34.640215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 10:37:34.641384 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 10:37:34.641384 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 10:37:34.641384 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 10:37:34.641384 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 10:37:34.641384 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 10:37:34.641384 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 10:37:34.641384 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 10:37:34.641384 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 10:37:34.641384 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 10:37:34.641384 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 10:37:34.641384 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 10:37:34.641384 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 10:37:34.641384 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 10:37:34.641384 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 10:37:34.641384 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 10:37:34.641384 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 10:37:34.643298 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 10:37:34.643298 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application with Lock( Sep 11 10:37:34.643298 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 10:37:34.643298 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self._enter() Sep 11 10:37:34.643298 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 10:37:34.643298 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 10:37:34.643298 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 10:37:34.643298 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application return self.creator() Sep 11 10:37:34.643298 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 10:37:34.643298 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application created_value = creator( Sep 11 10:37:34.643298 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 10:37:34.643298 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 10:37:34.643298 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 11 10:37:34.643298 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: ERROR keystone.server.flask.application Sep 11 10:37:34.643298 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 647/1304] 10.222.0.15 () {56 vars in 1117 bytes} [Sun Sep 11 10:37:34 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 10:37:34.647786 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b760662-0d73-4f53-a489-024fbed57f89 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:34.720846 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:37:34.743129 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b760662-0d73-4f53-a489-024fbed57f89 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:34.775685 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac6ca6d-43de-487d-aeab-123f2e66e2ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:34.776274 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac6ca6d-43de-487d-aeab-123f2e66e2ea None admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:34.776551 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac6ca6d-43de-487d-aeab-123f2e66e2ea None admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:34.776819 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac6ca6d-43de-487d-aeab-123f2e66e2ea None admin] PATH_INFO: `/v3/services` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:34.777428 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac6ca6d-43de-487d-aeab-123f2e66e2ea None admin] RBAC: Adding query filter params (name=placement) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:37:34.777727 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac6ca6d-43de-487d-aeab-123f2e66e2ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:37:34.779075 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac6ca6d-43de-487d-aeab-123f2e66e2ea None admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:37:34.784604 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 658/1305] 10.222.0.15 () {56 vars in 1116 bytes} [Sun Sep 11 10:37:34 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:37:34.790250 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9a7452-4909-40f6-819d-a4d5fbc76aab None admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:37:34.886146 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9a7452-4909-40f6-819d-a4d5fbc76aab None admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:37:34.917931 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89f9939-bebf-42e8-99f2-84a1d13a35b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:37:34.918511 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89f9939-bebf-42e8-99f2-84a1d13a35b8 None admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:37:34.918806 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89f9939-bebf-42e8-99f2-84a1d13a35b8 None admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:37:34.919056 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89f9939-bebf-42e8-99f2-84a1d13a35b8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:37:34.919747 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89f9939-bebf-42e8-99f2-84a1d13a35b8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:37:34.921091 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89f9939-bebf-42e8-99f2-84a1d13a35b8 None admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:37:34.940361 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 648/1306] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:37:34 2022] POST /identity/v3/endpoints => generated 331 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:37:44.781101 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=30 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:37:44.938795 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:38:38.798443 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9618afb-ca99-46c0-8987-27126cc81e01 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:38:38.798919 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9618afb-ca99-46c0-8987-27126cc81e01 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:38:38.799514 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9618afb-ca99-46c0-8987-27126cc81e01 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:38:38.806935 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d2ed68-a693-4ba0-afc9-477476ecfcb0 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:38:38.807343 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d2ed68-a693-4ba0-afc9-477476ecfcb0 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:38:38.807783 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d2ed68-a693-4ba0-afc9-477476ecfcb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:38:38.810884 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 659/1307] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:38:38 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 10:38:38.988589 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-64d2ed68-a693-4ba0-afc9-477476ecfcb0 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:38:38.989957 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-64d2ed68-a693-4ba0-afc9-477476ecfcb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:38:39.223162 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 649/1308] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:38:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:38:39.538033 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978d90d6-25af-4e5a-8e14-619e7e63db9d None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:38:39.538348 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978d90d6-25af-4e5a-8e14-619e7e63db9d None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:38:39.538623 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978d90d6-25af-4e5a-8e14-619e7e63db9d None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:38:39.540267 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 660/1309] 10.222.0.15 () {54 vars in 965 bytes} [Sun Sep 11 10:38:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:38:39.548609 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2f79b1-a919-4227-be07-515deaa8e007 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:38:39.548903 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2f79b1-a919-4227-be07-515deaa8e007 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:38:39.549189 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2f79b1-a919-4227-be07-515deaa8e007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:38:39.665728 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-ab2f79b1-a919-4227-be07-515deaa8e007 None None] MFA Rules not processed for user `40072bc8dcb5454382a93e20b63f8594`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:38:39.667891 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-ab2f79b1-a919-4227-be07-515deaa8e007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:38:39.696194 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:38:39.775440 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 650/1310] 10.222.0.15 () {58 vars in 1062 bytes} [Sun Sep 11 10:38:39 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 10:38:39.779377 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978d90d6-25af-4e5a-8e14-619e7e63db9d None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:38:39.782090 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-978d90d6-25af-4e5a-8e14-619e7e63db9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:38:39.903108 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978d90d6-25af-4e5a-8e14-619e7e63db9d None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:38:39.985313 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7cf6e8-a6b5-4bc6-940e-71a7f0ffb9b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:38:39.985932 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7cf6e8-a6b5-4bc6-940e-71a7f0ffb9b8 service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:38:39.986207 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7cf6e8-a6b5-4bc6-940e-71a7f0ffb9b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:38:39.986474 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7cf6e8-a6b5-4bc6-940e-71a7f0ffb9b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:38:39.988277 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-2a7cf6e8-a6b5-4bc6-940e-71a7f0ffb9b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:38:40.053098 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:38:40.118517 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7cf6e8-a6b5-4bc6-940e-71a7f0ffb9b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:38:40.120146 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7cf6e8-a6b5-4bc6-940e-71a7f0ffb9b8 service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:38:40.159649 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 661/1311] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:38:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:38:42.708663 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b8b61e-ab71-435c-9501-425ff2cde61d None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:38:42.709192 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b8b61e-ab71-435c-9501-425ff2cde61d None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:38:42.709192 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b8b61e-ab71-435c-9501-425ff2cde61d None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:38:42.710490 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 651/1312] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:38:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:38:42.715819 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949f2c8f-f74d-41e7-afa4-a3b4a72cac36 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:38:42.716220 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949f2c8f-f74d-41e7-afa4-a3b4a72cac36 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:38:42.716374 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949f2c8f-f74d-41e7-afa4-a3b4a72cac36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:38:42.787616 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-949f2c8f-f74d-41e7-afa4-a3b4a72cac36 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:38:42.788922 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-949f2c8f-f74d-41e7-afa4-a3b4a72cac36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:38:42.883033 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 662/1313] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:38:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:38:43.179272 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b8b61e-ab71-435c-9501-425ff2cde61d None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:38:43.180853 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-04b8b61e-ab71-435c-9501-425ff2cde61d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:38:43.287905 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b8b61e-ab71-435c-9501-425ff2cde61d None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:38:43.359933 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:38:43.362261 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d9921d-9fab-4b8b-8ba3-67e00e239e83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:38:43.363051 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d9921d-9fab-4b8b-8ba3-67e00e239e83 service neutron] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:38:43.363569 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d9921d-9fab-4b8b-8ba3-67e00e239e83 service neutron] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:38:43.364033 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d9921d-9fab-4b8b-8ba3-67e00e239e83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:38:43.365650 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-c3d9921d-9fab-4b8b-8ba3-67e00e239e83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:38:43.473173 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d9921d-9fab-4b8b-8ba3-67e00e239e83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:38:43.474522 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d9921d-9fab-4b8b-8ba3-67e00e239e83 service neutron] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:38:43.492867 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 652/1314] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:38:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:38:45.953603 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6044dd4c-9249-488e-8231-ee6f2aa72728 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:38:45.954147 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6044dd4c-9249-488e-8231-ee6f2aa72728 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:38:45.954259 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6044dd4c-9249-488e-8231-ee6f2aa72728 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:38:45.955538 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 663/1315] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:38:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:38:45.960778 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7580cbda-20bb-429b-96b0-c4cd880fae3f None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:38:45.961166 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7580cbda-20bb-429b-96b0-c4cd880fae3f None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:38:45.961322 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7580cbda-20bb-429b-96b0-c4cd880fae3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:38:46.027316 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-7580cbda-20bb-429b-96b0-c4cd880fae3f None None] MFA Rules not processed for user `0175a3d1c7824b08b476a3f905aad38e`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:38:46.028558 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-7580cbda-20bb-429b-96b0-c4cd880fae3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:38:46.080219 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:38:46.126899 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 653/1316] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:38:45 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:38:46.416143 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6044dd4c-9249-488e-8231-ee6f2aa72728 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:38:46.417691 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-6044dd4c-9249-488e-8231-ee6f2aa72728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:38:46.429527 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:38:46.522356 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6044dd4c-9249-488e-8231-ee6f2aa72728 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:38:46.593774 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d83173-36b0-4f5e-bf9f-72e90529d026 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:38:46.594310 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d83173-36b0-4f5e-bf9f-72e90529d026 service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:38:46.594586 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d83173-36b0-4f5e-bf9f-72e90529d026 service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:38:46.594854 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d83173-36b0-4f5e-bf9f-72e90529d026 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:38:46.596458 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-f0d83173-36b0-4f5e-bf9f-72e90529d026 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:38:46.658683 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:38:46.705987 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d83173-36b0-4f5e-bf9f-72e90529d026 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:38:46.707440 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d83173-36b0-4f5e-bf9f-72e90529d026 service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:38:46.725575 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 664/1317] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:38:46 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:38:49.453953 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d93c8d-141c-4c45-b4e7-6cd780d4b0b1 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:38:49.454567 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d93c8d-141c-4c45-b4e7-6cd780d4b0b1 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:38:49.454567 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d93c8d-141c-4c45-b4e7-6cd780d4b0b1 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:38:49.455879 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 654/1318] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:38:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:38:49.461111 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed5febe-3249-4de1-8625-4c7b56e31dff None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:38:49.461552 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed5febe-3249-4de1-8625-4c7b56e31dff None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:38:49.461655 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed5febe-3249-4de1-8625-4c7b56e31dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:38:49.527111 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-7ed5febe-3249-4de1-8625-4c7b56e31dff None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:38:49.528326 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-7ed5febe-3249-4de1-8625-4c7b56e31dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:38:49.621677 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 665/1319] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:38:49 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:38:49.909449 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d93c8d-141c-4c45-b4e7-6cd780d4b0b1 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:38:49.910981 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-32d93c8d-141c-4c45-b4e7-6cd780d4b0b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:38:50.013367 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d93c8d-141c-4c45-b4e7-6cd780d4b0b1 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:38:50.086370 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2e6f96-d7ee-4378-af7e-8b366c6cc834 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:38:50.086979 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2e6f96-d7ee-4378-af7e-8b366c6cc834 service neutron] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:38:50.087261 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2e6f96-d7ee-4378-af7e-8b366c6cc834 service neutron] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:38:50.087571 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2e6f96-d7ee-4378-af7e-8b366c6cc834 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:38:50.089172 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-cc2e6f96-d7ee-4378-af7e-8b366c6cc834 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:38:50.096706 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:38:50.198623 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2e6f96-d7ee-4378-af7e-8b366c6cc834 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:38:50.200012 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2e6f96-d7ee-4378-af7e-8b366c6cc834 service neutron] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:38:50.219730 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 655/1320] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:38:49 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:38:53.033916 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3075ad15-b021-49f5-af52-27cb1f9372dc None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:38:53.034796 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3075ad15-b021-49f5-af52-27cb1f9372dc None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:38:53.035129 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3075ad15-b021-49f5-af52-27cb1f9372dc None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:38:53.036943 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 666/1321] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:38:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:38:53.042352 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e5c977-6e3a-4eb3-a97e-08553db1e3ff None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:38:53.042790 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e5c977-6e3a-4eb3-a97e-08553db1e3ff None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:38:53.042931 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e5c977-6e3a-4eb3-a97e-08553db1e3ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:38:53.118700 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-77e5c977-6e3a-4eb3-a97e-08553db1e3ff None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:38:53.119955 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-77e5c977-6e3a-4eb3-a97e-08553db1e3ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:38:53.182781 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:38:53.228610 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 656/1322] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:38:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:38:53.515255 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3075ad15-b021-49f5-af52-27cb1f9372dc None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:38:53.516838 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-3075ad15-b021-49f5-af52-27cb1f9372dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:38:53.537864 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:38:53.622068 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3075ad15-b021-49f5-af52-27cb1f9372dc None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:38:53.694495 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb1f75c-9892-4be7-b0d2-237ac55987d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:38:53.695099 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb1f75c-9892-4be7-b0d2-237ac55987d9 service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:38:53.695482 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb1f75c-9892-4be7-b0d2-237ac55987d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:38:53.695813 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb1f75c-9892-4be7-b0d2-237ac55987d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:38:53.697503 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-8eb1f75c-9892-4be7-b0d2-237ac55987d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:38:53.774073 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:38:53.813251 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb1f75c-9892-4be7-b0d2-237ac55987d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:38:53.814802 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb1f75c-9892-4be7-b0d2-237ac55987d9 service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:38:53.833165 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 667/1323] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:38:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:38:56.048129 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc2a6e8-c13c-422c-a6d5-6e0e7fd6d53a None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:38:56.048766 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc2a6e8-c13c-422c-a6d5-6e0e7fd6d53a None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:38:56.048766 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc2a6e8-c13c-422c-a6d5-6e0e7fd6d53a None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:38:56.049994 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 657/1324] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:38:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:38:56.055109 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c12f0f-6fad-42c3-bfd7-67c65e5e95b8 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:38:56.055540 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c12f0f-6fad-42c3-bfd7-67c65e5e95b8 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:38:56.055696 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c12f0f-6fad-42c3-bfd7-67c65e5e95b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:38:56.121955 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-64c12f0f-6fad-42c3-bfd7-67c65e5e95b8 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:38:56.123150 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-64c12f0f-6fad-42c3-bfd7-67c65e5e95b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:38:56.230150 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 668/1325] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:38:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:38:56.513497 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc2a6e8-c13c-422c-a6d5-6e0e7fd6d53a None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:38:56.515353 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-6cc2a6e8-c13c-422c-a6d5-6e0e7fd6d53a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:38:56.618407 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc2a6e8-c13c-422c-a6d5-6e0e7fd6d53a None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:38:56.694114 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9631211-2f15-4cd3-bbb1-02a12b1b3141 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:38:56.694323 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9631211-2f15-4cd3-bbb1-02a12b1b3141 service neutron] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:38:56.694577 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9631211-2f15-4cd3-bbb1-02a12b1b3141 service neutron] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:38:56.694793 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9631211-2f15-4cd3-bbb1-02a12b1b3141 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:38:56.696522 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-a9631211-2f15-4cd3-bbb1-02a12b1b3141 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:38:56.716778 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:38:56.810458 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9631211-2f15-4cd3-bbb1-02a12b1b3141 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:38:56.812849 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9631211-2f15-4cd3-bbb1-02a12b1b3141 service neutron] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:38:56.830718 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 658/1326] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:38:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:38:59.057508 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a61c4-188f-4c40-956b-6746b8eb88dc None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:38:59.058023 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a61c4-188f-4c40-956b-6746b8eb88dc None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:38:59.058023 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a61c4-188f-4c40-956b-6746b8eb88dc None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:38:59.059359 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 669/1327] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:38:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:38:59.065189 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d45ed1-a38e-4272-bb78-3f59ac6894b0 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:38:59.065627 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d45ed1-a38e-4272-bb78-3f59ac6894b0 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:38:59.065735 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d45ed1-a38e-4272-bb78-3f59ac6894b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:38:59.136421 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-25d45ed1-a38e-4272-bb78-3f59ac6894b0 None None] MFA Rules not processed for user `0175a3d1c7824b08b476a3f905aad38e`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:38:59.137545 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-25d45ed1-a38e-4272-bb78-3f59ac6894b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:38:59.201558 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:38:59.237363 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 659/1328] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:38:59 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:38:59.594315 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6138958e-d7eb-4dbf-8601-1fe229fb57fa None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:38:59.594624 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6138958e-d7eb-4dbf-8601-1fe229fb57fa None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:38:59.595097 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6138958e-d7eb-4dbf-8601-1fe229fb57fa None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:38:59.596921 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 670/1329] 10.222.0.15 () {54 vars in 965 bytes} [Sun Sep 11 10:38:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:38:59.604657 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a553e1d-1a77-44c3-99b4-a04885ebb269 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:38:59.604941 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a553e1d-1a77-44c3-99b4-a04885ebb269 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:38:59.605214 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a553e1d-1a77-44c3-99b4-a04885ebb269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:38:59.694282 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-1a553e1d-1a77-44c3-99b4-a04885ebb269 None None] MFA Rules not processed for user `40072bc8dcb5454382a93e20b63f8594`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:38:59.695871 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-1a553e1d-1a77-44c3-99b4-a04885ebb269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:38:59.785921 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 660/1330] 10.222.0.15 () {58 vars in 1062 bytes} [Sun Sep 11 10:38:59 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:38:59.796109 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6138958e-d7eb-4dbf-8601-1fe229fb57fa None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:38:59.798799 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-6138958e-d7eb-4dbf-8601-1fe229fb57fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:38:59.839549 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:38:59.933093 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6138958e-d7eb-4dbf-8601-1fe229fb57fa None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:39:00.028251 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0647f6c-9c1d-4304-a4f9-e34fcff37da0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:39:00.028868 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0647f6c-9c1d-4304-a4f9-e34fcff37da0 service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:39:00.029146 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0647f6c-9c1d-4304-a4f9-e34fcff37da0 service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:39:00.029415 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0647f6c-9c1d-4304-a4f9-e34fcff37da0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:39:00.031872 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-c0647f6c-9c1d-4304-a4f9-e34fcff37da0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:39:00.117718 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:39:00.152143 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0647f6c-9c1d-4304-a4f9-e34fcff37da0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:39:00.153598 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0647f6c-9c1d-4304-a4f9-e34fcff37da0 service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:39:00.172991 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 671/1331] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:38:59 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:39:02.977244 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9006d145-30e1-462b-b929-c329c8e13b7f None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:39:02.977798 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9006d145-30e1-462b-b929-c329c8e13b7f None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:39:02.977798 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9006d145-30e1-462b-b929-c329c8e13b7f None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:39:02.979294 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 661/1332] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:39:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:39:02.984606 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6599acf1-e28f-40c0-a071-7bf465ed4c05 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:39:02.985053 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6599acf1-e28f-40c0-a071-7bf465ed4c05 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:39:02.985149 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6599acf1-e28f-40c0-a071-7bf465ed4c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:39:03.049770 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-6599acf1-e28f-40c0-a071-7bf465ed4c05 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:39:03.050955 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-6599acf1-e28f-40c0-a071-7bf465ed4c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:39:03.147839 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 672/1333] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:39:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 10:39:03.429810 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9006d145-30e1-462b-b929-c329c8e13b7f None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:39:03.431332 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-9006d145-30e1-462b-b929-c329c8e13b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:39:03.503073 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:39:03.537112 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9006d145-30e1-462b-b929-c329c8e13b7f None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:39:03.611551 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96697c05-08ef-448c-ba45-cced2a131cc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:39:03.612087 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96697c05-08ef-448c-ba45-cced2a131cc7 service neutron] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:39:03.612365 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96697c05-08ef-448c-ba45-cced2a131cc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:39:03.612633 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96697c05-08ef-448c-ba45-cced2a131cc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:39:03.614204 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-96697c05-08ef-448c-ba45-cced2a131cc7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:39:03.721851 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96697c05-08ef-448c-ba45-cced2a131cc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:39:03.723274 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96697c05-08ef-448c-ba45-cced2a131cc7 service neutron] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:39:03.733404 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:39:03.742119 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 662/1334] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:39:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:39:05.923483 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4781e3c9-a08c-4384-a9a8-80b4ae051a64 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:39:05.924056 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4781e3c9-a08c-4384-a9a8-80b4ae051a64 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:39:05.924056 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4781e3c9-a08c-4384-a9a8-80b4ae051a64 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:39:05.925369 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 673/1335] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:39:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:39:05.930687 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c2e4ae-f64a-4528-b4a8-4ec988fbb9c9 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:39:05.931094 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c2e4ae-f64a-4528-b4a8-4ec988fbb9c9 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:39:05.931225 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c2e4ae-f64a-4528-b4a8-4ec988fbb9c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:39:05.996844 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-74c2e4ae-f64a-4528-b4a8-4ec988fbb9c9 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:39:05.998043 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-74c2e4ae-f64a-4528-b4a8-4ec988fbb9c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:39:06.095001 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 663/1336] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:39:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:39:06.381840 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4781e3c9-a08c-4384-a9a8-80b4ae051a64 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:39:06.383367 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-4781e3c9-a08c-4384-a9a8-80b4ae051a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:39:06.423062 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:39:06.486566 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4781e3c9-a08c-4384-a9a8-80b4ae051a64 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:39:06.559280 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea9e27f-5cf9-4705-8b58-5d6b28842e31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:39:06.559970 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea9e27f-5cf9-4705-8b58-5d6b28842e31 service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:39:06.560198 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea9e27f-5cf9-4705-8b58-5d6b28842e31 service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:39:06.560468 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea9e27f-5cf9-4705-8b58-5d6b28842e31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:39:06.562592 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-8ea9e27f-5cf9-4705-8b58-5d6b28842e31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:39:06.644373 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:39:06.670541 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea9e27f-5cf9-4705-8b58-5d6b28842e31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:39:06.672018 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea9e27f-5cf9-4705-8b58-5d6b28842e31 service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:39:06.690267 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 674/1337] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:39:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:39:09.290537 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9982aae-bd52-4fad-98ba-3e82120aea90 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:39:09.291069 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9982aae-bd52-4fad-98ba-3e82120aea90 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:39:09.291069 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9982aae-bd52-4fad-98ba-3e82120aea90 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:39:09.292440 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 664/1338] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:39:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:39:09.298347 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a083965f-5216-44fb-84f6-66784fa28b05 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:39:09.298757 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a083965f-5216-44fb-84f6-66784fa28b05 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:39:09.298888 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a083965f-5216-44fb-84f6-66784fa28b05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:39:09.366348 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-a083965f-5216-44fb-84f6-66784fa28b05 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:39:09.367564 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-a083965f-5216-44fb-84f6-66784fa28b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:39:09.460854 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 675/1339] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:39:09 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:39:09.742050 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9982aae-bd52-4fad-98ba-3e82120aea90 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:39:09.743619 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-c9982aae-bd52-4fad-98ba-3e82120aea90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:39:09.818141 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:39:09.845838 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9982aae-bd52-4fad-98ba-3e82120aea90 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:39:09.919441 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284dead1-f69c-4239-a73e-284734bf94f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:39:09.920591 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284dead1-f69c-4239-a73e-284734bf94f3 service neutron] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:39:09.920868 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284dead1-f69c-4239-a73e-284734bf94f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:39:09.921136 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284dead1-f69c-4239-a73e-284734bf94f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:39:09.922715 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-284dead1-f69c-4239-a73e-284734bf94f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:39:10.030766 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284dead1-f69c-4239-a73e-284734bf94f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:39:10.032215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284dead1-f69c-4239-a73e-284734bf94f3 service neutron] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:39:10.050742 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 665/1340] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:39:09 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:39:14.223444 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e33e9eb-238f-42b4-a3b7-009675840303 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:39:14.223984 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e33e9eb-238f-42b4-a3b7-009675840303 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:39:14.223984 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e33e9eb-238f-42b4-a3b7-009675840303 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:39:14.225282 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 676/1341] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:39:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:39:14.230708 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaf455f-99bf-4972-8e9d-1d222a51603a None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:39:14.231096 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaf455f-99bf-4972-8e9d-1d222a51603a None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:39:14.231256 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaf455f-99bf-4972-8e9d-1d222a51603a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:39:14.236932 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:39:14.301650 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-3eaf455f-99bf-4972-8e9d-1d222a51603a None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:39:14.302847 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-3eaf455f-99bf-4972-8e9d-1d222a51603a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:39:14.400957 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 666/1342] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:39:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:39:14.688686 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e33e9eb-238f-42b4-a3b7-009675840303 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:39:14.690249 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-1e33e9eb-238f-42b4-a3b7-009675840303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:39:14.741940 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:39:14.799303 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e33e9eb-238f-42b4-a3b7-009675840303 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:39:14.875468 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a506039-126c-4ee7-b043-dc9cf2606238 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:39:14.876085 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a506039-126c-4ee7-b043-dc9cf2606238 service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:39:14.876365 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a506039-126c-4ee7-b043-dc9cf2606238 service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:39:14.876637 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a506039-126c-4ee7-b043-dc9cf2606238 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:39:14.878208 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-5a506039-126c-4ee7-b043-dc9cf2606238 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:39:14.973100 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:39:14.990666 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a506039-126c-4ee7-b043-dc9cf2606238 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:39:14.992146 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a506039-126c-4ee7-b043-dc9cf2606238 service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:39:15.010808 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 677/1343] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:39:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:39:17.780250 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab18a23-3d46-4aac-8b5f-51ad388a6f21 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:39:17.780813 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab18a23-3d46-4aac-8b5f-51ad388a6f21 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:39:17.780813 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab18a23-3d46-4aac-8b5f-51ad388a6f21 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:39:17.782123 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 667/1344] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:39:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:39:17.787271 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e254e-f50a-47ef-a585-2152f305ea35 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:39:17.787739 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e254e-f50a-47ef-a585-2152f305ea35 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:39:17.787865 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e254e-f50a-47ef-a585-2152f305ea35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:39:17.853048 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-e42e254e-f50a-47ef-a585-2152f305ea35 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:39:17.854508 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-e42e254e-f50a-47ef-a585-2152f305ea35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:39:17.952389 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 678/1345] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:39:17 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:39:18.236995 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab18a23-3d46-4aac-8b5f-51ad388a6f21 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:39:18.238514 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-aab18a23-3d46-4aac-8b5f-51ad388a6f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:39:18.330729 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:39:18.351395 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab18a23-3d46-4aac-8b5f-51ad388a6f21 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:39:18.429388 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0068bb59-2894-4b9d-9915-a26cd4017ee4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:39:18.429997 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0068bb59-2894-4b9d-9915-a26cd4017ee4 service neutron] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:39:18.430277 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0068bb59-2894-4b9d-9915-a26cd4017ee4 service neutron] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:39:18.430548 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0068bb59-2894-4b9d-9915-a26cd4017ee4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:39:18.432153 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-0068bb59-2894-4b9d-9915-a26cd4017ee4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:39:18.543387 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0068bb59-2894-4b9d-9915-a26cd4017ee4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:39:18.545461 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0068bb59-2894-4b9d-9915-a26cd4017ee4 service neutron] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:39:18.564290 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 668/1346] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:39:18 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:39:22.697818 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71433e36-ccb5-4695-a7c1-995fdcb855c9 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:39:22.699231 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71433e36-ccb5-4695-a7c1-995fdcb855c9 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:39:22.699740 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71433e36-ccb5-4695-a7c1-995fdcb855c9 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:39:22.703607 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 679/1347] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:39:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 10:39:22.708363 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beba69f-631c-49c6-83c5-487469714c86 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:39:22.709161 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beba69f-631c-49c6-83c5-487469714c86 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:39:22.710135 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beba69f-631c-49c6-83c5-487469714c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:39:22.728311 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:39:22.798021 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-0beba69f-631c-49c6-83c5-487469714c86 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:39:22.799253 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-0beba69f-631c-49c6-83c5-487469714c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:39:22.912674 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 669/1348] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:39:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:39:23.204400 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71433e36-ccb5-4695-a7c1-995fdcb855c9 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:39:23.205921 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-71433e36-ccb5-4695-a7c1-995fdcb855c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:39:23.274046 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:39:23.336448 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71433e36-ccb5-4695-a7c1-995fdcb855c9 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:39:23.409672 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981e71ef-dd7f-4d3b-82ba-d3e2df713c0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:39:23.410306 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e71ef-dd7f-4d3b-82ba-d3e2df713c0e service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:39:23.410587 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e71ef-dd7f-4d3b-82ba-d3e2df713c0e service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:39:23.410856 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e71ef-dd7f-4d3b-82ba-d3e2df713c0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:39:23.412481 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-981e71ef-dd7f-4d3b-82ba-d3e2df713c0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:39:23.516028 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:39:23.520002 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981e71ef-dd7f-4d3b-82ba-d3e2df713c0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:39:23.521426 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981e71ef-dd7f-4d3b-82ba-d3e2df713c0e service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:39:23.539823 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 680/1349] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:39:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:39:28.540945 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7028cf-3058-4a39-ac9d-8a1344b88586 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:39:28.541410 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7028cf-3058-4a39-ac9d-8a1344b88586 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:39:28.542058 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7028cf-3058-4a39-ac9d-8a1344b88586 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:39:28.551452 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 670/1350] 10.222.0.15 () {54 vars in 921 bytes} [Sun Sep 11 10:39:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:39:28.552130 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741ba339-99d5-4976-bca4-20b284cd5c9f None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:39:28.552130 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741ba339-99d5-4976-bca4-20b284cd5c9f None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:39:28.552130 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741ba339-99d5-4976-bca4-20b284cd5c9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:39:28.658881 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-741ba339-99d5-4976-bca4-20b284cd5c9f None None] MFA Rules not processed for user `08477b4c88f04c9e97391aba401879e0`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:39:28.660131 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-741ba339-99d5-4976-bca4-20b284cd5c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:39:28.786454 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 681/1351] 10.222.0.15 () {58 vars in 1018 bytes} [Sun Sep 11 10:39:28 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:39:32.909428 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=59 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:39:33.112741 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad50e0d-ef97-4443-a08c-32e352041edb None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:39:33.113088 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad50e0d-ef97-4443-a08c-32e352041edb None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:39:33.113289 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad50e0d-ef97-4443-a08c-32e352041edb None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:39:33.114793 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 671/1352] 10.222.0.15 () {54 vars in 921 bytes} [Sun Sep 11 10:39:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:39:33.119667 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c709daee-e7e7-4ef0-9955-64af1514b607 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:39:33.120525 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c709daee-e7e7-4ef0-9955-64af1514b607 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:39:33.121003 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c709daee-e7e7-4ef0-9955-64af1514b607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:39:33.181565 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:39:33.209196 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-c709daee-e7e7-4ef0-9955-64af1514b607 None None] MFA Rules not processed for user `08477b4c88f04c9e97391aba401879e0`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:39:33.210759 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-c709daee-e7e7-4ef0-9955-64af1514b607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:39:33.348080 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 682/1353] 10.222.0.15 () {58 vars in 1018 bytes} [Sun Sep 11 10:39:33 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:39:34.803153 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0958de0a-516a-41f9-b992-c08190f8b242 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:39:34.803743 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0958de0a-516a-41f9-b992-c08190f8b242 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:39:34.803743 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0958de0a-516a-41f9-b992-c08190f8b242 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:39:34.805334 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 672/1354] 10.222.0.15 () {54 vars in 921 bytes} [Sun Sep 11 10:39:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:39:34.810815 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22abe454-fc83-46ae-8c5a-9a7be4edc72e None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:39:34.811433 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22abe454-fc83-46ae-8c5a-9a7be4edc72e None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:39:34.811433 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22abe454-fc83-46ae-8c5a-9a7be4edc72e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:39:34.900267 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-22abe454-fc83-46ae-8c5a-9a7be4edc72e None None] MFA Rules not processed for user `08477b4c88f04c9e97391aba401879e0`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:39:34.901372 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-22abe454-fc83-46ae-8c5a-9a7be4edc72e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:39:34.945490 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:39:35.020421 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 683/1355] 10.222.0.15 () {58 vars in 1018 bytes} [Sun Sep 11 10:39:34 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 10:39:38.441580 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75638dd9-1106-4b86-a653-b7de0c6e24a7 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:39:38.442397 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75638dd9-1106-4b86-a653-b7de0c6e24a7 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:39:38.442835 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75638dd9-1106-4b86-a653-b7de0c6e24a7 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:39:38.444363 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 673/1356] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:39:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:39:38.449823 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a010be3-f5e7-4906-94b2-2868c7430a4a None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:39:38.451374 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a010be3-f5e7-4906-94b2-2868c7430a4a None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:39:38.452041 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a010be3-f5e7-4906-94b2-2868c7430a4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:39:38.533392 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-2a010be3-f5e7-4906-94b2-2868c7430a4a None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:39:38.534742 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-2a010be3-f5e7-4906-94b2-2868c7430a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:39:38.636884 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 684/1357] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:39:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:39:38.945598 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1460229-262c-4843-816a-ed9527c87fa7 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:39:38.946650 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1460229-262c-4843-816a-ed9527c87fa7 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:39:38.947532 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1460229-262c-4843-816a-ed9527c87fa7 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:39:38.951655 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 674/1358] 10.222.0.15 () {54 vars in 962 bytes} [Sun Sep 11 10:39:38 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:39:38.959382 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1100e4a-cb27-4184-ade3-b3708605bdec None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:39:38.960106 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1100e4a-cb27-4184-ade3-b3708605bdec None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:39:38.960807 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1100e4a-cb27-4184-ade3-b3708605bdec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:39:38.980390 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:39:39.053773 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-e1100e4a-cb27-4184-ade3-b3708605bdec None None] MFA Rules not processed for user `2bd5356829b6454993bab9bad2f79fed`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:39:39.055130 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-e1100e4a-cb27-4184-ade3-b3708605bdec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:39:39.180182 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 685/1359] 10.222.0.15 () {58 vars in 1059 bytes} [Sun Sep 11 10:39:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:39:39.189859 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1460229-262c-4843-816a-ed9527c87fa7 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:39:39.191746 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-e1460229-262c-4843-816a-ed9527c87fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:39:39.388171 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1460229-262c-4843-816a-ed9527c87fa7 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:39:39.470446 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:39:39.512516 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54250a6f-b27c-4488-b57a-d8e286ee9edd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bd5356829b6454993bab9bad2f79fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:39:39.514150 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54250a6f-b27c-4488-b57a-d8e286ee9edd service nova] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:39:39.514543 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54250a6f-b27c-4488-b57a-d8e286ee9edd service nova] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:39:39.514912 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54250a6f-b27c-4488-b57a-d8e286ee9edd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:39:39.516653 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-54250a6f-b27c-4488-b57a-d8e286ee9edd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:39:39.621228 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54250a6f-b27c-4488-b57a-d8e286ee9edd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:39:39.623025 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54250a6f-b27c-4488-b57a-d8e286ee9edd service nova] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:39:39.643574 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 675/1360] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:39:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:39:42.094777 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ee9f9-3b93-4b0d-9c70-6d053e9012b2 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:39:42.095271 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ee9f9-3b93-4b0d-9c70-6d053e9012b2 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:39:42.095726 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ee9f9-3b93-4b0d-9c70-6d053e9012b2 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:39:42.096900 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 686/1361] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:39:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:39:42.105335 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6624ce6-e23e-417d-865c-bcdc3076f020 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:39:42.105777 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6624ce6-e23e-417d-865c-bcdc3076f020 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:39:42.107519 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6624ce6-e23e-417d-865c-bcdc3076f020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:39:42.194162 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-b6624ce6-e23e-417d-865c-bcdc3076f020 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:39:42.195654 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-b6624ce6-e23e-417d-865c-bcdc3076f020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:39:42.238806 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:39:42.312925 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 676/1362] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:39:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:39:42.609076 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4ee9f9-3b93-4b0d-9c70-6d053e9012b2 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:39:42.610628 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-6b4ee9f9-3b93-4b0d-9c70-6d053e9012b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:39:42.689354 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:39:42.804495 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4ee9f9-3b93-4b0d-9c70-6d053e9012b2 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:39:42.964044 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dc99e1-e485-4345-a55d-bf926878fa4a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bd5356829b6454993bab9bad2f79fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:39:42.964664 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dc99e1-e485-4345-a55d-bf926878fa4a service nova] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:39:42.964942 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dc99e1-e485-4345-a55d-bf926878fa4a service nova] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:39:42.965215 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dc99e1-e485-4345-a55d-bf926878fa4a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:39:42.967226 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-e1dc99e1-e485-4345-a55d-bf926878fa4a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:39:42.992748 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:39:43.069354 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dc99e1-e485-4345-a55d-bf926878fa4a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:39:43.070977 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dc99e1-e485-4345-a55d-bf926878fa4a service nova] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:39:43.092286 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 687/1363] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:39:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:39:45.633458 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32dad40-8c94-485e-a37b-005309dd8e39 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:39:45.634066 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32dad40-8c94-485e-a37b-005309dd8e39 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:39:45.634066 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32dad40-8c94-485e-a37b-005309dd8e39 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:39:45.635849 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 677/1364] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:39:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:39:45.641953 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e3cde-433b-4d47-a69a-bc02c5375218 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:39:45.642769 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e3cde-433b-4d47-a69a-bc02c5375218 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:39:45.643227 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e3cde-433b-4d47-a69a-bc02c5375218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:39:45.714846 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-2f8e3cde-433b-4d47-a69a-bc02c5375218 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:39:45.716483 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-2f8e3cde-433b-4d47-a69a-bc02c5375218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:39:45.823927 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:39:45.835899 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 688/1365] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:39:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:39:46.121810 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beca0f7-2379-415d-97a6-4f19032cf796 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:39:46.122762 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beca0f7-2379-415d-97a6-4f19032cf796 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:39:46.123138 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beca0f7-2379-415d-97a6-4f19032cf796 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:39:46.124796 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 678/1366] 10.222.0.15 () {54 vars in 962 bytes} [Sun Sep 11 10:39:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:39:46.130205 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3938810e-dd62-43ef-9392-d25f63c028cd None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:39:46.130494 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3938810e-dd62-43ef-9392-d25f63c028cd None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:39:46.130774 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3938810e-dd62-43ef-9392-d25f63c028cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:39:46.212611 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-3938810e-dd62-43ef-9392-d25f63c028cd None None] MFA Rules not processed for user `2bd5356829b6454993bab9bad2f79fed`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:39:46.213826 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-3938810e-dd62-43ef-9392-d25f63c028cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:39:46.347884 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 689/1367] 10.222.0.15 () {58 vars in 1059 bytes} [Sun Sep 11 10:39:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:39:46.356970 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beca0f7-2379-415d-97a6-4f19032cf796 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:39:46.358695 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-7beca0f7-2379-415d-97a6-4f19032cf796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:39:46.522528 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beca0f7-2379-415d-97a6-4f19032cf796 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:39:46.546119 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:39:46.655767 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f35075-269f-42c3-bfce-644e78d31373 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bd5356829b6454993bab9bad2f79fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:39:46.656412 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f35075-269f-42c3-bfce-644e78d31373 service nova] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:39:46.656782 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f35075-269f-42c3-bfce-644e78d31373 service nova] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:39:46.657057 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f35075-269f-42c3-bfce-644e78d31373 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:39:46.658647 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-11f35075-269f-42c3-bfce-644e78d31373 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:39:46.770006 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f35075-269f-42c3-bfce-644e78d31373 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:39:46.771631 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f35075-269f-42c3-bfce-644e78d31373 service nova] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:39:46.790853 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 679/1368] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:39:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:39:46.830515 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4256457-1059-4be4-847f-9efe80d43546 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:39:46.831091 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4256457-1059-4be4-847f-9efe80d43546 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:39:46.831091 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4256457-1059-4be4-847f-9efe80d43546 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:39:46.832708 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 690/1369] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:39:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:39:46.839311 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a3a227-ce87-483e-98b3-85482a654f89 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:39:46.839892 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a3a227-ce87-483e-98b3-85482a654f89 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:39:46.840455 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a3a227-ce87-483e-98b3-85482a654f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:39:46.876235 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:39:46.921191 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-77a3a227-ce87-483e-98b3-85482a654f89 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:39:46.922521 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-77a3a227-ce87-483e-98b3-85482a654f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:39:47.044857 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 680/1370] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:39:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:39:47.343181 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc12af70-b9af-47a8-a912-72892902f070 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:39:47.343632 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc12af70-b9af-47a8-a912-72892902f070 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:39:47.344122 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc12af70-b9af-47a8-a912-72892902f070 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:39:47.345695 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 691/1371] 10.222.0.15 () {54 vars in 964 bytes} [Sun Sep 11 10:39:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:39:47.350875 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f6a38d-5379-46de-a3e8-11fa2e6d3b36 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:39:47.351172 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f6a38d-5379-46de-a3e8-11fa2e6d3b36 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:39:47.351484 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f6a38d-5379-46de-a3e8-11fa2e6d3b36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:39:47.432609 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-c6f6a38d-5379-46de-a3e8-11fa2e6d3b36 None None] MFA Rules not processed for user `0bbb481abb8e43788e3412ac7d908ad3`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:39:47.433842 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-c6f6a38d-5379-46de-a3e8-11fa2e6d3b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:39:47.492417 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:39:47.522419 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 681/1372] 10.222.0.15 () {58 vars in 1061 bytes} [Sun Sep 11 10:39:47 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:39:47.532962 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc12af70-b9af-47a8-a912-72892902f070 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:39:47.534477 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-bc12af70-b9af-47a8-a912-72892902f070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:39:47.594787 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:39:47.654248 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc12af70-b9af-47a8-a912-72892902f070 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:39:47.737798 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b7a97c-271b-4cf6-85b6-eec9bf58b2b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:39:47.738603 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b7a97c-271b-4cf6-85b6-eec9bf58b2b7 service cinder] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:39:47.738891 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b7a97c-271b-4cf6-85b6-eec9bf58b2b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:39:47.739167 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b7a97c-271b-4cf6-85b6-eec9bf58b2b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:39:47.740807 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-e1b7a97c-271b-4cf6-85b6-eec9bf58b2b7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:39:47.861427 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:39:47.878057 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b7a97c-271b-4cf6-85b6-eec9bf58b2b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:39:47.879588 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b7a97c-271b-4cf6-85b6-eec9bf58b2b7 service cinder] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:39:47.904320 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 692/1373] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:39:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:39:49.062675 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ba108-c158-4049-8d13-abe02a2e0138 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:39:49.063213 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ba108-c158-4049-8d13-abe02a2e0138 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:39:49.063213 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ba108-c158-4049-8d13-abe02a2e0138 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:39:49.064581 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 682/1374] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:39:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:39:49.069745 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ff40d-24fb-43c0-badb-2fd5bd869f79 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:39:49.070185 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ff40d-24fb-43c0-badb-2fd5bd869f79 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:39:49.070383 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ff40d-24fb-43c0-badb-2fd5bd869f79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:39:49.136813 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-c93ff40d-24fb-43c0-badb-2fd5bd869f79 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:39:49.138627 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-c93ff40d-24fb-43c0-badb-2fd5bd869f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:39:49.235926 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 693/1375] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:39:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:39:49.534711 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ba108-c158-4049-8d13-abe02a2e0138 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:39:49.536321 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-5a9ba108-c158-4049-8d13-abe02a2e0138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:39:49.695710 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ba108-c158-4049-8d13-abe02a2e0138 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:39:49.750768 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:39:49.813877 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8810178f-8ae9-4da3-a3b9-a72e595e11e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bd5356829b6454993bab9bad2f79fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:39:49.814485 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8810178f-8ae9-4da3-a3b9-a72e595e11e9 service nova] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:39:49.814755 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8810178f-8ae9-4da3-a3b9-a72e595e11e9 service nova] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:39:49.815028 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8810178f-8ae9-4da3-a3b9-a72e595e11e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:39:49.816654 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-8810178f-8ae9-4da3-a3b9-a72e595e11e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:39:49.912396 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8810178f-8ae9-4da3-a3b9-a72e595e11e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:39:49.914007 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8810178f-8ae9-4da3-a3b9-a72e595e11e9 service nova] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:39:49.933558 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 683/1376] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:39:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:39:52.175927 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd71f290-39ee-4a36-b2f5-8f2886dd0530 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:39:52.176534 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd71f290-39ee-4a36-b2f5-8f2886dd0530 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:39:52.176534 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd71f290-39ee-4a36-b2f5-8f2886dd0530 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:39:52.177992 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 694/1377] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:39:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 10:39:52.182976 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc933bb8-28ed-4805-8078-2e52811b5bb1 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:39:52.183376 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc933bb8-28ed-4805-8078-2e52811b5bb1 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:39:52.183621 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc933bb8-28ed-4805-8078-2e52811b5bb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:39:52.254081 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-fc933bb8-28ed-4805-8078-2e52811b5bb1 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:39:52.255415 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-fc933bb8-28ed-4805-8078-2e52811b5bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:39:52.263008 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:39:52.372532 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 684/1378] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:39:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:39:52.679553 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd71f290-39ee-4a36-b2f5-8f2886dd0530 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:39:52.681273 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-dd71f290-39ee-4a36-b2f5-8f2886dd0530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:39:52.749708 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:39:52.839062 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd71f290-39ee-4a36-b2f5-8f2886dd0530 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:39:52.957810 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c563d8a-21ed-481a-8795-394e4c407460 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bd5356829b6454993bab9bad2f79fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:39:52.958425 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c563d8a-21ed-481a-8795-394e4c407460 service nova] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:39:52.958707 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c563d8a-21ed-481a-8795-394e4c407460 service nova] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:39:52.958982 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c563d8a-21ed-481a-8795-394e4c407460 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:39:52.960595 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-4c563d8a-21ed-481a-8795-394e4c407460 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:39:52.979980 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:39:53.054882 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c563d8a-21ed-481a-8795-394e4c407460 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:39:53.056548 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c563d8a-21ed-481a-8795-394e4c407460 service nova] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:39:53.076046 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 695/1379] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:39:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:39:55.295561 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b862e159-4f9b-478d-bbe9-405ee270c117 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:39:55.296159 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b862e159-4f9b-478d-bbe9-405ee270c117 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:39:55.296159 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b862e159-4f9b-478d-bbe9-405ee270c117 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:39:55.297517 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 685/1380] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:39:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:39:55.303047 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce58ed30-f155-4d8b-8031-9940dcd4e510 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:39:55.303496 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce58ed30-f155-4d8b-8031-9940dcd4e510 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:39:55.303650 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce58ed30-f155-4d8b-8031-9940dcd4e510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:39:55.375554 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-ce58ed30-f155-4d8b-8031-9940dcd4e510 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:39:55.377177 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-ce58ed30-f155-4d8b-8031-9940dcd4e510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:39:55.478570 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:39:55.500272 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 696/1381] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:39:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:39:55.814842 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b862e159-4f9b-478d-bbe9-405ee270c117 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:39:55.816479 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-b862e159-4f9b-478d-bbe9-405ee270c117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:39:55.965591 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:39:55.979768 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b862e159-4f9b-478d-bbe9-405ee270c117 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:39:56.107308 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045c48c8-80ed-4483-afb9-a26167ca9e14 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bd5356829b6454993bab9bad2f79fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:39:56.108066 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045c48c8-80ed-4483-afb9-a26167ca9e14 service nova] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:39:56.108417 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045c48c8-80ed-4483-afb9-a26167ca9e14 service nova] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:39:56.108764 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045c48c8-80ed-4483-afb9-a26167ca9e14 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:39:56.110808 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-045c48c8-80ed-4483-afb9-a26167ca9e14 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:39:56.227171 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045c48c8-80ed-4483-afb9-a26167ca9e14 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:39:56.228823 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045c48c8-80ed-4483-afb9-a26167ca9e14 service nova] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:39:56.239316 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:39:56.249030 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 686/1382] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:39:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:39:59.029523 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e65e58a-0d02-408f-a256-deb535f5c351 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:39:59.030073 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e65e58a-0d02-408f-a256-deb535f5c351 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:39:59.030073 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e65e58a-0d02-408f-a256-deb535f5c351 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:39:59.031408 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 697/1383] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:39:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:39:59.038180 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4da339-1bef-45ba-b19b-ed1303e03308 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:39:59.038661 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4da339-1bef-45ba-b19b-ed1303e03308 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:39:59.038774 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4da339-1bef-45ba-b19b-ed1303e03308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:39:59.106353 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-3e4da339-1bef-45ba-b19b-ed1303e03308 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:39:59.107765 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-3e4da339-1bef-45ba-b19b-ed1303e03308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:39:59.207581 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 687/1384] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:39:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:39:59.498216 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e65e58a-0d02-408f-a256-deb535f5c351 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:39:59.499886 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-6e65e58a-0d02-408f-a256-deb535f5c351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:39:59.673326 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e65e58a-0d02-408f-a256-deb535f5c351 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:39:59.759262 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:39:59.807504 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97bffef-1e6f-4210-ab0e-a7501858f98f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bd5356829b6454993bab9bad2f79fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:39:59.808122 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97bffef-1e6f-4210-ab0e-a7501858f98f service nova] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:39:59.808401 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97bffef-1e6f-4210-ab0e-a7501858f98f service nova] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:39:59.808698 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97bffef-1e6f-4210-ab0e-a7501858f98f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:39:59.810812 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-c97bffef-1e6f-4210-ab0e-a7501858f98f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:39:59.907542 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97bffef-1e6f-4210-ab0e-a7501858f98f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:39:59.909491 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97bffef-1e6f-4210-ab0e-a7501858f98f service nova] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:39:59.928327 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 698/1385] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:39:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:40:02.329040 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e544f7a0-f8fb-4456-af02-b2a4bf6b093e None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:40:02.329560 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e544f7a0-f8fb-4456-af02-b2a4bf6b093e None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:40:02.329560 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e544f7a0-f8fb-4456-af02-b2a4bf6b093e None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:40:02.330923 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 688/1386] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:40:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:40:02.336262 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea4d69-6538-4802-8d5f-4ea336e668f1 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:40:02.336697 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea4d69-6538-4802-8d5f-4ea336e668f1 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:40:02.336826 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea4d69-6538-4802-8d5f-4ea336e668f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:40:02.403367 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-f5ea4d69-6538-4802-8d5f-4ea336e668f1 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:40:02.404598 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-f5ea4d69-6538-4802-8d5f-4ea336e668f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:40:02.437390 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:40:02.502542 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 699/1387] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:40:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:40:02.795383 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e544f7a0-f8fb-4456-af02-b2a4bf6b093e None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:40:02.796938 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-e544f7a0-f8fb-4456-af02-b2a4bf6b093e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:40:02.875714 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:40:02.947915 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e544f7a0-f8fb-4456-af02-b2a4bf6b093e None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:40:03.063878 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcc7fa0-6910-4d90-a9dd-a2ad548a4f76 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bd5356829b6454993bab9bad2f79fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:40:03.064406 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcc7fa0-6910-4d90-a9dd-a2ad548a4f76 service nova] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:40:03.064687 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcc7fa0-6910-4d90-a9dd-a2ad548a4f76 service nova] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:40:03.064959 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcc7fa0-6910-4d90-a9dd-a2ad548a4f76 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:40:03.066554 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-2bcc7fa0-6910-4d90-a9dd-a2ad548a4f76 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:40:03.105642 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:40:03.159431 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcc7fa0-6910-4d90-a9dd-a2ad548a4f76 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:40:03.161093 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcc7fa0-6910-4d90-a9dd-a2ad548a4f76 service nova] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:40:03.182805 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 689/1388] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:40:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:40:03.289591 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898eb0ad-e33d-4800-b5fa-a6845843b8e0 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:40:03.289874 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898eb0ad-e33d-4800-b5fa-a6845843b8e0 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:40:03.290152 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898eb0ad-e33d-4800-b5fa-a6845843b8e0 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:40:03.291663 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 700/1389] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:40:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:40:03.298442 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1950f2d9-ed42-4699-8dcc-23f2a78ddfe6 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:40:03.298763 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1950f2d9-ed42-4699-8dcc-23f2a78ddfe6 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:40:03.299085 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1950f2d9-ed42-4699-8dcc-23f2a78ddfe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:40:03.376020 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-1950f2d9-ed42-4699-8dcc-23f2a78ddfe6 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:40:03.377180 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-1950f2d9-ed42-4699-8dcc-23f2a78ddfe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:40:03.480463 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 690/1390] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:40:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:40:03.792741 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc25889-2ef9-47c2-a6b3-c04857161876 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:40:03.793309 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc25889-2ef9-47c2-a6b3-c04857161876 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:40:03.793426 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc25889-2ef9-47c2-a6b3-c04857161876 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:40:03.795632 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 701/1391] 10.222.0.15 () {54 vars in 966 bytes} [Sun Sep 11 10:40:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:40:03.801046 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c381d0-fdd7-4704-a8c0-e3e378574c0f None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:40:03.801553 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c381d0-fdd7-4704-a8c0-e3e378574c0f None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:40:03.802036 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c381d0-fdd7-4704-a8c0-e3e378574c0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:40:03.819350 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:40:03.891498 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-a6c381d0-fdd7-4704-a8c0-e3e378574c0f None None] MFA Rules not processed for user `1632da5fbe384b5c849be1d4d86c0f1d`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:40:03.893114 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-a6c381d0-fdd7-4704-a8c0-e3e378574c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:40:03.981189 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 691/1392] 10.222.0.15 () {58 vars in 1063 bytes} [Sun Sep 11 10:40:03 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:40:03.990415 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc25889-2ef9-47c2-a6b3-c04857161876 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:40:03.992019 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-ebc25889-2ef9-47c2-a6b3-c04857161876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:40:04.098157 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc25889-2ef9-47c2-a6b3-c04857161876 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:40:04.162116 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:40:04.178347 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7631b0-9505-48f5-ae57-797adf928539 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1632da5fbe384b5c849be1d4d86c0f1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:40:04.178956 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7631b0-9505-48f5-ae57-797adf928539 service glance] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:40:04.179341 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7631b0-9505-48f5-ae57-797adf928539 service glance] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:40:04.179654 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7631b0-9505-48f5-ae57-797adf928539 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:40:04.181263 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-3e7631b0-9505-48f5-ae57-797adf928539 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:40:04.416233 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7631b0-9505-48f5-ae57-797adf928539 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:40:04.417773 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7631b0-9505-48f5-ae57-797adf928539 service glance] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:40:04.436886 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 702/1393] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:40:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:40:05.206278 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af72d659-d074-4931-b75b-04de1257f524 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:40:05.206811 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af72d659-d074-4931-b75b-04de1257f524 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:40:05.207039 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af72d659-d074-4931-b75b-04de1257f524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:40:05.276720 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-af72d659-d074-4931-b75b-04de1257f524 None None] MFA Rules not processed for user `bb44e7aef0c7436d93a7233307b5f095`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:40:05.278192 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-af72d659-d074-4931-b75b-04de1257f524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:40:05.337282 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:40:05.373000 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 692/1394] 10.222.0.15 () {58 vars in 1009 bytes} [Sun Sep 11 10:40:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:40:05.415977 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fedc289-4a07-47c7-a00e-98e945842fbd None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:40:05.416512 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fedc289-4a07-47c7-a00e-98e945842fbd None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:40:05.416512 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fedc289-4a07-47c7-a00e-98e945842fbd None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:40:05.417827 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 703/1395] 10.222.0.15 () {54 vars in 943 bytes} [Sun Sep 11 10:40:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:40:05.425133 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d412415-561b-4242-aa7b-21ee75a839f8 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:40:05.425519 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d412415-561b-4242-aa7b-21ee75a839f8 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:40:05.425797 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d412415-561b-4242-aa7b-21ee75a839f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:40:05.440107 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bbf823-2fa5-43d8-8ab9-e42775f0800f None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:40:05.440374 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bbf823-2fa5-43d8-8ab9-e42775f0800f None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:40:05.440651 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bbf823-2fa5-43d8-8ab9-e42775f0800f None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:40:05.442240 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 704/1396] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:40:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 10:40:05.447258 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86520473-b692-41c2-9b4a-2f67b9ccdf6d None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:40:05.447565 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86520473-b692-41c2-9b4a-2f67b9ccdf6d None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:40:05.448106 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86520473-b692-41c2-9b4a-2f67b9ccdf6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:40:05.511009 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-9d412415-561b-4242-aa7b-21ee75a839f8 None None] MFA Rules not processed for user `cc7ca566e66a45b29e3d02f47bf84981`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:40:05.512333 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-9d412415-561b-4242-aa7b-21ee75a839f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:40:05.518238 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-86520473-b692-41c2-9b4a-2f67b9ccdf6d None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:40:05.519403 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-86520473-b692-41c2-9b4a-2f67b9ccdf6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:40:05.561234 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:40:05.618070 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 705/1397] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:40:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:40:05.625522 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 693/1398] 10.222.0.15 () {58 vars in 1040 bytes} [Sun Sep 11 10:40:05 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:40:05.635559 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86520473-b692-41c2-9b4a-2f67b9ccdf6d None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:40:05.636754 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-86520473-b692-41c2-9b4a-2f67b9ccdf6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:40:05.748434 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86520473-b692-41c2-9b4a-2f67b9ccdf6d None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:40:05.812179 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:40:05.852611 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497777bf-7386-4420-952e-686005c5e70a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7ca566e66a45b29e3d02f47bf84981', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:40:05.853217 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497777bf-7386-4420-952e-686005c5e70a service swift] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:40:05.853490 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497777bf-7386-4420-952e-686005c5e70a service swift] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:40:05.853751 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497777bf-7386-4420-952e-686005c5e70a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:40:05.856322 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-497777bf-7386-4420-952e-686005c5e70a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:40:05.914395 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d412415-561b-4242-aa7b-21ee75a839f8 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:40:05.916051 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-9d412415-561b-4242-aa7b-21ee75a839f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:40:05.940715 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497777bf-7386-4420-952e-686005c5e70a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:40:05.942296 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497777bf-7386-4420-952e-686005c5e70a service swift] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:40:05.950775 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:40:05.952240 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 706/1399] 10.222.0.15 () {60 vars in 1384 bytes} [Sun Sep 11 10:40:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 10:40:06.026827 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d412415-561b-4242-aa7b-21ee75a839f8 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:40:06.172946 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49a379d-ee31-4137-9480-5d94c823f7ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bd5356829b6454993bab9bad2f79fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:40:06.173551 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49a379d-ee31-4137-9480-5d94c823f7ab service nova] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:40:06.173828 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49a379d-ee31-4137-9480-5d94c823f7ab service nova] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:40:06.174091 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49a379d-ee31-4137-9480-5d94c823f7ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:40:06.175705 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-b49a379d-ee31-4137-9480-5d94c823f7ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:40:06.223372 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:40:06.277390 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49a379d-ee31-4137-9480-5d94c823f7ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:40:06.279787 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49a379d-ee31-4137-9480-5d94c823f7ab service nova] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:40:06.298477 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 694/1400] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:40:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 10:40:08.533290 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ccc178-f998-4f37-b9de-9e681618941d None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:40:08.533836 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ccc178-f998-4f37-b9de-9e681618941d None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:40:08.533836 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ccc178-f998-4f37-b9de-9e681618941d None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:40:08.535466 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 707/1401] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:40:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:40:08.540685 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dee0bd-87a9-4626-89c8-8c62e20915d2 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:40:08.541095 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dee0bd-87a9-4626-89c8-8c62e20915d2 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:40:08.541239 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dee0bd-87a9-4626-89c8-8c62e20915d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:40:08.613949 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-e0dee0bd-87a9-4626-89c8-8c62e20915d2 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:40:08.615079 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-e0dee0bd-87a9-4626-89c8-8c62e20915d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:40:08.755468 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 695/1402] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:40:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:40:09.046048 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ccc178-f998-4f37-b9de-9e681618941d None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:40:09.048284 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-73ccc178-f998-4f37-b9de-9e681618941d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:40:09.153197 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:40:09.211281 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ccc178-f998-4f37-b9de-9e681618941d None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:40:09.346825 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d7aa75-8d11-4a0d-88c9-6d4c1a88129d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bd5356829b6454993bab9bad2f79fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:40:09.347578 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d7aa75-8d11-4a0d-88c9-6d4c1a88129d service nova] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:40:09.347936 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d7aa75-8d11-4a0d-88c9-6d4c1a88129d service nova] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:40:09.348297 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d7aa75-8d11-4a0d-88c9-6d4c1a88129d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:40:09.350209 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-89d7aa75-8d11-4a0d-88c9-6d4c1a88129d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:40:09.431270 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:40:09.460423 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d7aa75-8d11-4a0d-88c9-6d4c1a88129d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:40:09.464753 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d7aa75-8d11-4a0d-88c9-6d4c1a88129d service nova] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:40:09.484903 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 708/1403] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:40:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:40:11.722620 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce38c53-3f7a-4e00-a27e-095c6b8bf68d None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:40:11.723153 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce38c53-3f7a-4e00-a27e-095c6b8bf68d None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:40:11.723153 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce38c53-3f7a-4e00-a27e-095c6b8bf68d None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:40:11.727796 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 696/1404] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:40:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 10:40:11.730208 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab0b933-6763-4fc7-8083-de03f065fb69 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:40:11.731222 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab0b933-6763-4fc7-8083-de03f065fb69 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:40:11.731441 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab0b933-6763-4fc7-8083-de03f065fb69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:40:11.805003 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-5ab0b933-6763-4fc7-8083-de03f065fb69 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:40:11.806207 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-5ab0b933-6763-4fc7-8083-de03f065fb69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:40:11.915464 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 709/1405] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:40:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:40:12.216906 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce38c53-3f7a-4e00-a27e-095c6b8bf68d None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:40:12.218525 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-7ce38c53-3f7a-4e00-a27e-095c6b8bf68d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:40:12.249071 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:40:12.410685 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce38c53-3f7a-4e00-a27e-095c6b8bf68d None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:40:12.516572 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:40:12.541003 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7cf747-1b77-436e-ba5b-1dee96f58e4d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bd5356829b6454993bab9bad2f79fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:40:12.541965 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7cf747-1b77-436e-ba5b-1dee96f58e4d service nova] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:40:12.542437 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7cf747-1b77-436e-ba5b-1dee96f58e4d service nova] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:40:12.542775 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7cf747-1b77-436e-ba5b-1dee96f58e4d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:40:12.545337 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-7c7cf747-1b77-436e-ba5b-1dee96f58e4d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:40:12.644078 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7cf747-1b77-436e-ba5b-1dee96f58e4d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:40:12.645692 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7cf747-1b77-436e-ba5b-1dee96f58e4d service nova] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:40:12.667309 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 697/1406] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:40:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:40:14.900890 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79d9129-c811-435d-9741-2777cd680713 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:40:14.901469 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79d9129-c811-435d-9741-2777cd680713 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:40:14.901469 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79d9129-c811-435d-9741-2777cd680713 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:40:14.908222 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ba31c-a5a3-4fec-8d35-ffba71269aed None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:40:14.908736 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ba31c-a5a3-4fec-8d35-ffba71269aed None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:40:14.908834 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ba31c-a5a3-4fec-8d35-ffba71269aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:40:14.913129 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 710/1407] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:40:14 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:40:14.982541 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-1c8ba31c-a5a3-4fec-8d35-ffba71269aed None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:40:14.985821 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-1c8ba31c-a5a3-4fec-8d35-ffba71269aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:40:15.048353 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:40:15.099592 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 698/1408] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:40:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 10:40:15.394918 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79d9129-c811-435d-9741-2777cd680713 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:40:15.396530 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-b79d9129-c811-435d-9741-2777cd680713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:40:15.442629 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:40:15.553353 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79d9129-c811-435d-9741-2777cd680713 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:40:15.678351 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15edeb14-48b0-4de5-9de4-c192073fb33f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bd5356829b6454993bab9bad2f79fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:40:15.678915 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15edeb14-48b0-4de5-9de4-c192073fb33f service nova] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:40:15.679191 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15edeb14-48b0-4de5-9de4-c192073fb33f service nova] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:40:15.679489 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15edeb14-48b0-4de5-9de4-c192073fb33f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:40:15.681075 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-15edeb14-48b0-4de5-9de4-c192073fb33f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:40:15.683555 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:40:15.788682 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15edeb14-48b0-4de5-9de4-c192073fb33f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:40:15.790937 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15edeb14-48b0-4de5-9de4-c192073fb33f service nova] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:40:15.810545 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 711/1409] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:40:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:40:25.097053 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=19 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:40:25.807349 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=44 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:40:28.473940 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e80919-c3e4-47e5-846a-e9d8762abea1 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:40:28.474523 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e80919-c3e4-47e5-846a-e9d8762abea1 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:40:28.474523 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e80919-c3e4-47e5-846a-e9d8762abea1 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:40:28.475904 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 699/1410] 10.222.0.15 () {54 vars in 918 bytes} [Sun Sep 11 10:40:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:40:28.481831 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9648dc6c-9eb3-4ed9-9bbb-8a74dde39f8b None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:40:28.482280 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9648dc6c-9eb3-4ed9-9bbb-8a74dde39f8b None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:40:28.482384 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9648dc6c-9eb3-4ed9-9bbb-8a74dde39f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:40:28.557289 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-9648dc6c-9eb3-4ed9-9bbb-8a74dde39f8b None None] MFA Rules not processed for user `08477b4c88f04c9e97391aba401879e0`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:40:28.558518 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-9648dc6c-9eb3-4ed9-9bbb-8a74dde39f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:40:28.661680 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 712/1411] 10.222.0.15 () {58 vars in 1015 bytes} [Sun Sep 11 10:40:28 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:40:28.676560 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1733e0-9721-458a-9db9-3b84d319387a None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:40:28.676843 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1733e0-9721-458a-9db9-3b84d319387a None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:40:28.677115 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1733e0-9721-458a-9db9-3b84d319387a None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:40:28.678428 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 700/1412] 10.222.0.15 () {54 vars in 918 bytes} [Sun Sep 11 10:40:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:40:28.683254 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3ad5d3-07ba-4ce4-bc79-b63272357361 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:40:28.683609 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3ad5d3-07ba-4ce4-bc79-b63272357361 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:40:28.683932 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3ad5d3-07ba-4ce4-bc79-b63272357361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:40:28.755618 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:40:28.758326 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-3a3ad5d3-07ba-4ce4-bc79-b63272357361 None None] MFA Rules not processed for user `2bd5356829b6454993bab9bad2f79fed`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:40:28.760213 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-3a3ad5d3-07ba-4ce4-bc79-b63272357361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:40:28.875338 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 713/1413] 10.222.0.15 () {58 vars in 1015 bytes} [Sun Sep 11 10:40:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:40:31.446749 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed79c9-8ae1-4f38-92d5-c18b66875df8 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:40:31.447308 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed79c9-8ae1-4f38-92d5-c18b66875df8 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:40:31.447308 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed79c9-8ae1-4f38-92d5-c18b66875df8 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:40:31.448738 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 701/1414] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 10:40:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:40:31.454150 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe510b-47b1-4e56-813f-cabf34116eb5 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:40:31.454582 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe510b-47b1-4e56-813f-cabf34116eb5 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:40:31.454689 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe510b-47b1-4e56-813f-cabf34116eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:40:31.519902 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-14fe510b-47b1-4e56-813f-cabf34116eb5 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:40:31.521092 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-14fe510b-47b1-4e56-813f-cabf34116eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:40:31.577259 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:40:31.617970 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 714/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 10:40:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:40:31.906216 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fed79c9-8ae1-4f38-92d5-c18b66875df8 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:40:31.907765 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-0fed79c9-8ae1-4f38-92d5-c18b66875df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:40:32.056539 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fed79c9-8ae1-4f38-92d5-c18b66875df8 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:40:32.129214 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:40:32.169907 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8dbed1-cd43-4016-a280-39cebbb4e2fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bd5356829b6454993bab9bad2f79fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:40:32.170437 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8dbed1-cd43-4016-a280-39cebbb4e2fc service nova] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:40:32.170710 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8dbed1-cd43-4016-a280-39cebbb4e2fc service nova] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:40:32.170976 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8dbed1-cd43-4016-a280-39cebbb4e2fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:40:32.172623 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-9e8dbed1-cd43-4016-a280-39cebbb4e2fc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:40:32.260819 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8dbed1-cd43-4016-a280-39cebbb4e2fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:40:32.263043 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8dbed1-cd43-4016-a280-39cebbb4e2fc service nova] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:40:32.280561 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 702/1416] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:40:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:40:33.417123 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555b15c-b21f-4bda-8d67-c664a4893187 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:40:33.418267 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555b15c-b21f-4bda-8d67-c664a4893187 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:40:33.418267 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555b15c-b21f-4bda-8d67-c664a4893187 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:40:33.419692 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 715/1417] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 10:40:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 10:40:33.425786 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e272be-0143-40dc-ac5b-a3186a8eb1c1 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:40:33.426243 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e272be-0143-40dc-ac5b-a3186a8eb1c1 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:40:33.426378 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e272be-0143-40dc-ac5b-a3186a8eb1c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:40:33.502244 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-e8e272be-0143-40dc-ac5b-a3186a8eb1c1 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:40:33.503471 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-e8e272be-0143-40dc-ac5b-a3186a8eb1c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:40:33.531746 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:40:33.601912 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 703/1418] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 10:40:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:40:33.751491 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b555b15c-b21f-4bda-8d67-c664a4893187 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:40:33.752991 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-b555b15c-b21f-4bda-8d67-c664a4893187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:40:33.898262 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b555b15c-b21f-4bda-8d67-c664a4893187 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:40:33.937993 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:40:34.010266 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b0773e-39a2-4bf2-9b93-1fa0a4f7942e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bd5356829b6454993bab9bad2f79fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:40:34.010865 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0773e-39a2-4bf2-9b93-1fa0a4f7942e service nova] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:40:34.011717 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0773e-39a2-4bf2-9b93-1fa0a4f7942e service nova] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:40:34.011991 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0773e-39a2-4bf2-9b93-1fa0a4f7942e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:40:34.013603 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-43b0773e-39a2-4bf2-9b93-1fa0a4f7942e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:40:34.101256 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b0773e-39a2-4bf2-9b93-1fa0a4f7942e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:40:34.102844 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b0773e-39a2-4bf2-9b93-1fa0a4f7942e service nova] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:40:34.122052 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 716/1419] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:40:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:40:35.265832 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d26ca-940e-4492-9bab-a6587ced7488 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:40:35.266410 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d26ca-940e-4492-9bab-a6587ced7488 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:40:35.266410 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d26ca-940e-4492-9bab-a6587ced7488 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:40:35.267734 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 704/1420] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 10:40:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:40:35.273806 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0116b7c7-898b-4bf1-9a9f-2fbd430986ac None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:40:35.274243 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0116b7c7-898b-4bf1-9a9f-2fbd430986ac None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:40:35.274380 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0116b7c7-898b-4bf1-9a9f-2fbd430986ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:40:35.332324 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:40:35.355590 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-0116b7c7-898b-4bf1-9a9f-2fbd430986ac None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:40:35.356909 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-0116b7c7-898b-4bf1-9a9f-2fbd430986ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:40:35.452894 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 717/1421] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 10:40:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:40:35.608976 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534d26ca-940e-4492-9bab-a6587ced7488 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:40:35.610412 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-534d26ca-940e-4492-9bab-a6587ced7488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:40:35.755302 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534d26ca-940e-4492-9bab-a6587ced7488 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:40:35.767739 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:40:35.869519 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95b9acf-58f2-41be-a969-724234c66fec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bd5356829b6454993bab9bad2f79fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:40:35.870030 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95b9acf-58f2-41be-a969-724234c66fec service nova] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:40:35.870333 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95b9acf-58f2-41be-a969-724234c66fec service nova] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:40:35.871441 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95b9acf-58f2-41be-a969-724234c66fec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:40:35.873205 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-a95b9acf-58f2-41be-a969-724234c66fec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:40:35.961734 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95b9acf-58f2-41be-a969-724234c66fec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:40:35.963389 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95b9acf-58f2-41be-a969-724234c66fec service nova] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:40:35.982651 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 705/1422] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:40:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:40:37.095356 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e40dd4-8a07-4f3c-9ea2-2623417d06ef None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:40:37.095946 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e40dd4-8a07-4f3c-9ea2-2623417d06ef None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:40:37.095946 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e40dd4-8a07-4f3c-9ea2-2623417d06ef None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:40:37.097314 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 718/1423] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 10:40:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:40:37.103352 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5508e3-345f-40d6-9365-2946b8fa1b64 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:40:37.103796 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5508e3-345f-40d6-9365-2946b8fa1b64 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:40:37.103939 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5508e3-345f-40d6-9365-2946b8fa1b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:40:37.124649 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:40:37.181415 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-fb5508e3-345f-40d6-9365-2946b8fa1b64 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:40:37.182632 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-fb5508e3-345f-40d6-9365-2946b8fa1b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:40:37.280381 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 706/1424] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 10:40:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:40:37.431050 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e40dd4-8a07-4f3c-9ea2-2623417d06ef None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:40:37.432484 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-69e40dd4-8a07-4f3c-9ea2-2623417d06ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:40:37.542094 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:40:37.577765 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e40dd4-8a07-4f3c-9ea2-2623417d06ef None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:40:37.688257 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395ebf09-e8ff-485a-bf63-7c9dd06317e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bd5356829b6454993bab9bad2f79fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:40:37.688791 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395ebf09-e8ff-485a-bf63-7c9dd06317e4 service nova] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:40:37.689071 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395ebf09-e8ff-485a-bf63-7c9dd06317e4 service nova] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:40:37.689344 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395ebf09-e8ff-485a-bf63-7c9dd06317e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:40:37.690924 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-395ebf09-e8ff-485a-bf63-7c9dd06317e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:40:37.765767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:40:37.780378 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395ebf09-e8ff-485a-bf63-7c9dd06317e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:40:37.781981 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395ebf09-e8ff-485a-bf63-7c9dd06317e4 service nova] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:40:37.800156 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 719/1425] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:40:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:40:38.913203 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee337c-fe9a-49cf-a545-deb44c513de6 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:40:38.913765 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee337c-fe9a-49cf-a545-deb44c513de6 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:40:38.913765 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee337c-fe9a-49cf-a545-deb44c513de6 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:40:38.915076 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 707/1426] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 10:40:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:40:38.922153 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2978c3-ab79-41f0-a49b-3ae5e435d3a9 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:40:38.922619 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2978c3-ab79-41f0-a49b-3ae5e435d3a9 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:40:38.922716 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2978c3-ab79-41f0-a49b-3ae5e435d3a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:40:38.999286 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-9a2978c3-ab79-41f0-a49b-3ae5e435d3a9 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:40:39.000519 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-9a2978c3-ab79-41f0-a49b-3ae5e435d3a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:40:39.098225 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 720/1427] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 10:40:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:40:39.252185 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acee337c-fe9a-49cf-a545-deb44c513de6 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:40:39.253725 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-acee337c-fe9a-49cf-a545-deb44c513de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:40:39.341230 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:40:39.404804 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acee337c-fe9a-49cf-a545-deb44c513de6 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:40:39.517406 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab60834-a860-451a-861f-94bcf2fc948d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bd5356829b6454993bab9bad2f79fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:40:39.518010 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab60834-a860-451a-861f-94bcf2fc948d service nova] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:40:39.518289 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab60834-a860-451a-861f-94bcf2fc948d service nova] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:40:39.518561 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab60834-a860-451a-861f-94bcf2fc948d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:40:39.520175 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-dab60834-a860-451a-861f-94bcf2fc948d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:40:39.571411 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:40:39.612891 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab60834-a860-451a-861f-94bcf2fc948d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:40:39.614465 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab60834-a860-451a-861f-94bcf2fc948d service nova] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:40:39.632652 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 708/1428] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:40:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:40:40.687176 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae3a27a-c4aa-4881-a828-3e8077182047 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:40:40.687176 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae3a27a-c4aa-4881-a828-3e8077182047 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:40:40.687176 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae3a27a-c4aa-4881-a828-3e8077182047 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:40:40.688553 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 721/1429] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 10:40:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:40:40.694588 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a981eab5-27a9-4e90-9ddb-08466df1390b None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:40:40.695054 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a981eab5-27a9-4e90-9ddb-08466df1390b None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:40:40.695159 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a981eab5-27a9-4e90-9ddb-08466df1390b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:40:40.773840 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-a981eab5-27a9-4e90-9ddb-08466df1390b None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:40:40.775033 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-a981eab5-27a9-4e90-9ddb-08466df1390b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:40:40.875935 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 709/1430] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 10:40:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:40:41.028370 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae3a27a-c4aa-4881-a828-3e8077182047 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:40:41.029905 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-5ae3a27a-c4aa-4881-a828-3e8077182047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:40:41.078008 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:40:41.172884 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae3a27a-c4aa-4881-a828-3e8077182047 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:40:41.284646 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d531065-1fdb-4327-8ef4-73d921a6d875 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bd5356829b6454993bab9bad2f79fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:40:41.285174 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d531065-1fdb-4327-8ef4-73d921a6d875 service nova] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:40:41.285453 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d531065-1fdb-4327-8ef4-73d921a6d875 service nova] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:40:41.285722 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d531065-1fdb-4327-8ef4-73d921a6d875 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:40:41.287282 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-0d531065-1fdb-4327-8ef4-73d921a6d875 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:40:41.294891 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:40:41.378044 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d531065-1fdb-4327-8ef4-73d921a6d875 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:40:41.379694 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d531065-1fdb-4327-8ef4-73d921a6d875 service nova] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:40:41.397250 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 722/1431] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:40:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:40:42.462689 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fdc427-3c88-458b-9bc3-f151639e582f None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:40:42.463249 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fdc427-3c88-458b-9bc3-f151639e582f None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:40:42.463249 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fdc427-3c88-458b-9bc3-f151639e582f None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:40:42.465333 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 710/1432] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 10:40:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:40:42.471579 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834b68df-a7af-40dd-87aa-e1f4629029ba None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:40:42.472020 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834b68df-a7af-40dd-87aa-e1f4629029ba None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:40:42.472123 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834b68df-a7af-40dd-87aa-e1f4629029ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:40:42.548251 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-834b68df-a7af-40dd-87aa-e1f4629029ba None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:40:42.549439 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-834b68df-a7af-40dd-87aa-e1f4629029ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:40:42.616098 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:40:42.649992 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 723/1433] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 10:40:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:40:42.801665 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fdc427-3c88-458b-9bc3-f151639e582f None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:40:42.803169 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-a8fdc427-3c88-458b-9bc3-f151639e582f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:40:42.820526 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:40:42.949602 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fdc427-3c88-458b-9bc3-f151639e582f None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:40:43.035295 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:40:43.061876 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c25b199-9727-42d1-b3dd-723bd1e337cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bd5356829b6454993bab9bad2f79fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:40:43.062484 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c25b199-9727-42d1-b3dd-723bd1e337cb service nova] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:40:43.062764 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c25b199-9727-42d1-b3dd-723bd1e337cb service nova] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:40:43.063034 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c25b199-9727-42d1-b3dd-723bd1e337cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:40:43.064642 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-8c25b199-9727-42d1-b3dd-723bd1e337cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:40:43.154293 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c25b199-9727-42d1-b3dd-723bd1e337cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:40:43.155914 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c25b199-9727-42d1-b3dd-723bd1e337cb service nova] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:40:43.174271 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 711/1434] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:40:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:40:44.279536 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d3bbcc-a04e-49a8-9d50-2b5b70038fad None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:40:44.280036 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d3bbcc-a04e-49a8-9d50-2b5b70038fad None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:40:44.280166 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d3bbcc-a04e-49a8-9d50-2b5b70038fad None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:40:44.281411 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 724/1435] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 10:40:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:40:44.287982 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0124bc-961b-4cfb-82c2-8e9dc04c99b6 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:40:44.288408 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0124bc-961b-4cfb-82c2-8e9dc04c99b6 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:40:44.288527 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0124bc-961b-4cfb-82c2-8e9dc04c99b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:40:44.369487 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-ee0124bc-961b-4cfb-82c2-8e9dc04c99b6 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:40:44.371435 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-ee0124bc-961b-4cfb-82c2-8e9dc04c99b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:40:44.411903 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:40:44.467269 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 712/1436] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 10:40:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:40:44.619326 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d3bbcc-a04e-49a8-9d50-2b5b70038fad None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:40:44.621531 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-95d3bbcc-a04e-49a8-9d50-2b5b70038fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:40:44.766746 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d3bbcc-a04e-49a8-9d50-2b5b70038fad None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:40:44.819933 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:40:44.878873 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a2e658-3e1f-4f41-8b35-707ec6d7f09f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bd5356829b6454993bab9bad2f79fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:40:44.879405 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a2e658-3e1f-4f41-8b35-707ec6d7f09f service nova] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:40:44.879711 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a2e658-3e1f-4f41-8b35-707ec6d7f09f service nova] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:40:44.879981 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a2e658-3e1f-4f41-8b35-707ec6d7f09f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:40:44.881575 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-d7a2e658-3e1f-4f41-8b35-707ec6d7f09f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:40:44.972589 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a2e658-3e1f-4f41-8b35-707ec6d7f09f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:40:44.974169 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a2e658-3e1f-4f41-8b35-707ec6d7f09f service nova] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:40:44.992702 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 725/1437] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:40:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:40:46.050551 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de782d8-6d8a-4173-aaf4-a96c9f79691e None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:40:46.051125 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de782d8-6d8a-4173-aaf4-a96c9f79691e None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:40:46.051125 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de782d8-6d8a-4173-aaf4-a96c9f79691e None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:40:46.052462 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 713/1438] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 10:40:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:40:46.058802 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daec8508-b9c1-4538-87bb-da64685632de None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:40:46.059228 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daec8508-b9c1-4538-87bb-da64685632de None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:40:46.059375 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daec8508-b9c1-4538-87bb-da64685632de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:40:46.134818 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-daec8508-b9c1-4538-87bb-da64685632de None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:40:46.137142 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-daec8508-b9c1-4538-87bb-da64685632de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:40:46.139117 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:40:46.235998 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 726/1439] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 10:40:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:40:46.384400 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de782d8-6d8a-4173-aaf4-a96c9f79691e None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:40:46.385977 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-9de782d8-6d8a-4173-aaf4-a96c9f79691e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:40:46.532143 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de782d8-6d8a-4173-aaf4-a96c9f79691e None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:40:46.558052 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:40:46.642916 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7262aad0-f352-4593-93a5-682c6a221b05 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bd5356829b6454993bab9bad2f79fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:40:46.643463 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7262aad0-f352-4593-93a5-682c6a221b05 service nova] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:40:46.643743 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7262aad0-f352-4593-93a5-682c6a221b05 service nova] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:40:46.644010 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7262aad0-f352-4593-93a5-682c6a221b05 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:40:46.645576 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-7262aad0-f352-4593-93a5-682c6a221b05 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:40:46.734095 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7262aad0-f352-4593-93a5-682c6a221b05 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:40:46.735719 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7262aad0-f352-4593-93a5-682c6a221b05 service nova] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:40:46.753282 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 714/1440] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:40:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:40:48.805956 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e879212-9911-4616-9868-c68b19a4a2d4 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:40:48.806518 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e879212-9911-4616-9868-c68b19a4a2d4 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:40:48.806518 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e879212-9911-4616-9868-c68b19a4a2d4 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:40:48.807892 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 727/1441] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 10:40:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:40:48.813417 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242ac7c5-da39-4dc6-bb3f-90889755bb27 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:40:48.813829 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242ac7c5-da39-4dc6-bb3f-90889755bb27 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:40:48.813962 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242ac7c5-da39-4dc6-bb3f-90889755bb27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:40:48.856060 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:40:48.884275 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-242ac7c5-da39-4dc6-bb3f-90889755bb27 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:40:48.886057 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-242ac7c5-da39-4dc6-bb3f-90889755bb27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:40:48.983327 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 715/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 10:40:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:40:49.269521 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e879212-9911-4616-9868-c68b19a4a2d4 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:40:49.271091 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-6e879212-9911-4616-9868-c68b19a4a2d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:40:49.373255 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e879212-9911-4616-9868-c68b19a4a2d4 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:40:49.405010 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:40:49.445318 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a2684a-9845-4f14-82ed-3acd4d65aeaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:40:49.445917 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a2684a-9845-4f14-82ed-3acd4d65aeaf service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:40:49.446192 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a2684a-9845-4f14-82ed-3acd4d65aeaf service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:40:49.446461 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a2684a-9845-4f14-82ed-3acd4d65aeaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:40:49.448091 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-d5a2684a-9845-4f14-82ed-3acd4d65aeaf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:40:49.554511 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a2684a-9845-4f14-82ed-3acd4d65aeaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:40:49.555971 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a2684a-9845-4f14-82ed-3acd4d65aeaf service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:40:49.574396 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 728/1443] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:40:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:40:51.883931 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e65c2c-a4c3-4816-9920-1d18c042f7b2 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:40:51.884511 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e65c2c-a4c3-4816-9920-1d18c042f7b2 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:40:51.884511 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e65c2c-a4c3-4816-9920-1d18c042f7b2 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:40:51.886331 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 716/1444] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 10:40:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:40:51.891581 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f2e962-a214-4709-9b7e-75fa352dd0f2 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:40:51.892157 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f2e962-a214-4709-9b7e-75fa352dd0f2 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:40:51.892157 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f2e962-a214-4709-9b7e-75fa352dd0f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:40:51.956969 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-95f2e962-a214-4709-9b7e-75fa352dd0f2 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:40:51.958171 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-95f2e962-a214-4709-9b7e-75fa352dd0f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:40:51.971441 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:40:52.054779 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 729/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 10:40:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:40:52.339085 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e65c2c-a4c3-4816-9920-1d18c042f7b2 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:40:52.340780 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-67e65c2c-a4c3-4816-9920-1d18c042f7b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:40:52.442935 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e65c2c-a4c3-4816-9920-1d18c042f7b2 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:40:52.455864 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:40:52.517879 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0374b7-42e8-436f-a3da-747528fb0f97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:40:52.518477 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0374b7-42e8-436f-a3da-747528fb0f97 service neutron] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:40:52.518752 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0374b7-42e8-436f-a3da-747528fb0f97 service neutron] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:40:52.519019 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0374b7-42e8-436f-a3da-747528fb0f97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:40:52.520605 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-4e0374b7-42e8-436f-a3da-747528fb0f97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:40:52.629067 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0374b7-42e8-436f-a3da-747528fb0f97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:40:52.630431 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0374b7-42e8-436f-a3da-747528fb0f97 service neutron] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:40:52.648623 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 717/1446] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:40:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:40:55.116152 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22a3257-7026-45bb-92b3-c95df16d5c8b None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:40:55.116729 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22a3257-7026-45bb-92b3-c95df16d5c8b None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:40:55.116729 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22a3257-7026-45bb-92b3-c95df16d5c8b None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:40:55.118037 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 730/1447] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 10:40:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:40:55.123953 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ce152b-375f-415c-a2cd-1ee602b6dfb8 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:40:55.124361 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ce152b-375f-415c-a2cd-1ee602b6dfb8 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:40:55.124522 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ce152b-375f-415c-a2cd-1ee602b6dfb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:40:55.173169 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:40:55.191532 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-a3ce152b-375f-415c-a2cd-1ee602b6dfb8 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:40:55.192749 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-a3ce152b-375f-415c-a2cd-1ee602b6dfb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:40:55.410032 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 718/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 10:40:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:40:55.692844 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22a3257-7026-45bb-92b3-c95df16d5c8b None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:40:55.694377 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-e22a3257-7026-45bb-92b3-c95df16d5c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:40:55.797578 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22a3257-7026-45bb-92b3-c95df16d5c8b None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:40:55.836708 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:40:55.870784 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe2f94-e0bc-454f-9cfb-379c23a11817 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:40:55.871388 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe2f94-e0bc-454f-9cfb-379c23a11817 service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:40:55.871694 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe2f94-e0bc-454f-9cfb-379c23a11817 service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:40:55.871967 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe2f94-e0bc-454f-9cfb-379c23a11817 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:40:55.873536 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-90fe2f94-e0bc-454f-9cfb-379c23a11817 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:40:55.982001 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fe2f94-e0bc-454f-9cfb-379c23a11817 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:40:55.983346 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fe2f94-e0bc-454f-9cfb-379c23a11817 service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:40:56.000989 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 731/1449] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:40:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:40:58.282515 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5fe8f9-3657-445e-8828-f2427e56c681 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:40:58.283084 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5fe8f9-3657-445e-8828-f2427e56c681 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:40:58.283084 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5fe8f9-3657-445e-8828-f2427e56c681 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:40:58.284422 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 719/1450] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 10:40:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:40:58.290115 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48deb64-c62f-459a-908a-832c59788637 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:40:58.290546 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48deb64-c62f-459a-908a-832c59788637 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:40:58.290648 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48deb64-c62f-459a-908a-832c59788637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:40:58.356848 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-d48deb64-c62f-459a-908a-832c59788637 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:40:58.358056 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-d48deb64-c62f-459a-908a-832c59788637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:40:58.379079 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:40:58.455134 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 732/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 10:40:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:40:58.741874 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5fe8f9-3657-445e-8828-f2427e56c681 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:40:58.743444 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-cb5fe8f9-3657-445e-8828-f2427e56c681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:40:58.845890 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5fe8f9-3657-445e-8828-f2427e56c681 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:40:58.868070 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:40:58.921801 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6761d4c6-4cb5-44ed-8c35-246e657a5882 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:40:58.922407 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6761d4c6-4cb5-44ed-8c35-246e657a5882 service neutron] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:40:58.922684 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6761d4c6-4cb5-44ed-8c35-246e657a5882 service neutron] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:40:58.922954 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6761d4c6-4cb5-44ed-8c35-246e657a5882 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:40:58.924683 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-6761d4c6-4cb5-44ed-8c35-246e657a5882 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:40:59.033547 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6761d4c6-4cb5-44ed-8c35-246e657a5882 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:40:59.034965 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6761d4c6-4cb5-44ed-8c35-246e657a5882 service neutron] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:40:59.052858 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 720/1452] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:40:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:41:01.328488 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e55a0-61f7-4d11-9519-3dacc596c02d None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:41:01.328921 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e55a0-61f7-4d11-9519-3dacc596c02d None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:41:01.329039 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e55a0-61f7-4d11-9519-3dacc596c02d None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:41:01.330389 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 733/1453] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 10:41:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:41:01.335922 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c056f380-4b17-483f-9494-33f5b3ac3921 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:41:01.336360 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c056f380-4b17-483f-9494-33f5b3ac3921 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:41:01.336459 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c056f380-4b17-483f-9494-33f5b3ac3921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:41:01.400340 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:41:01.404775 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-c056f380-4b17-483f-9494-33f5b3ac3921 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:41:01.405995 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-c056f380-4b17-483f-9494-33f5b3ac3921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:41:01.501491 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 721/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 10:41:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:41:01.785972 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94e55a0-61f7-4d11-9519-3dacc596c02d None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:41:01.787788 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-c94e55a0-61f7-4d11-9519-3dacc596c02d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:41:01.889250 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94e55a0-61f7-4d11-9519-3dacc596c02d None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:41:01.937860 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:41:01.962042 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d7a515-36c5-482e-9145-4edf9a7697d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:41:01.962646 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d7a515-36c5-482e-9145-4edf9a7697d8 service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:41:01.962919 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d7a515-36c5-482e-9145-4edf9a7697d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:41:01.963187 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d7a515-36c5-482e-9145-4edf9a7697d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:41:01.965345 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-52d7a515-36c5-482e-9145-4edf9a7697d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:41:02.071984 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d7a515-36c5-482e-9145-4edf9a7697d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:41:02.073602 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d7a515-36c5-482e-9145-4edf9a7697d8 service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:41:02.091789 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 734/1455] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:41:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:41:04.310645 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e0d5f0-2d72-4d3f-9e30-482b6578f0a6 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:41:04.311216 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e0d5f0-2d72-4d3f-9e30-482b6578f0a6 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:41:04.311216 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e0d5f0-2d72-4d3f-9e30-482b6578f0a6 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:41:04.312972 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 722/1456] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 10:41:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:41:04.317900 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6443eb-d18d-4a9d-b9d0-c5c0babf49ab None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:41:04.318318 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6443eb-d18d-4a9d-b9d0-c5c0babf49ab None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:41:04.318487 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6443eb-d18d-4a9d-b9d0-c5c0babf49ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:41:04.384837 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-1c6443eb-d18d-4a9d-b9d0-c5c0babf49ab None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:41:04.386764 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-1c6443eb-d18d-4a9d-b9d0-c5c0babf49ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:41:04.415000 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:41:04.482404 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 735/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 10:41:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:41:04.771121 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e0d5f0-2d72-4d3f-9e30-482b6578f0a6 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:41:04.772670 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-a5e0d5f0-2d72-4d3f-9e30-482b6578f0a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:41:04.874929 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e0d5f0-2d72-4d3f-9e30-482b6578f0a6 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:41:04.903544 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:41:04.948372 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1affd51d-883b-4765-930b-280ecd3cca0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:41:04.948964 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1affd51d-883b-4765-930b-280ecd3cca0e service neutron] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:41:04.949237 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1affd51d-883b-4765-930b-280ecd3cca0e service neutron] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:41:04.949505 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1affd51d-883b-4765-930b-280ecd3cca0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:41:04.951068 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-1affd51d-883b-4765-930b-280ecd3cca0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:41:05.056819 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1affd51d-883b-4765-930b-280ecd3cca0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:41:05.058871 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1affd51d-883b-4765-930b-280ecd3cca0e service neutron] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:41:05.076799 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 723/1458] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:41:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:41:07.285947 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895fc4b0-dacd-4d51-adae-0beb8c4841d4 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:41:07.286528 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895fc4b0-dacd-4d51-adae-0beb8c4841d4 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:41:07.286528 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895fc4b0-dacd-4d51-adae-0beb8c4841d4 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:41:07.287883 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 736/1459] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 10:41:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:41:07.293472 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72112bd7-986e-4a4d-bfde-0ad4eb44e9b4 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:41:07.293940 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72112bd7-986e-4a4d-bfde-0ad4eb44e9b4 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:41:07.294050 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72112bd7-986e-4a4d-bfde-0ad4eb44e9b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:41:07.361383 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-72112bd7-986e-4a4d-bfde-0ad4eb44e9b4 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:41:07.362589 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-72112bd7-986e-4a4d-bfde-0ad4eb44e9b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:41:07.371674 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:41:07.461373 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 724/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 10:41:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:41:07.751269 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895fc4b0-dacd-4d51-adae-0beb8c4841d4 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:41:07.752856 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-895fc4b0-dacd-4d51-adae-0beb8c4841d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:41:07.857560 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895fc4b0-dacd-4d51-adae-0beb8c4841d4 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:41:07.916022 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:41:07.934833 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15c1b8a-3b54-44ad-acff-1c91afa63f4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:41:07.935451 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c1b8a-3b54-44ad-acff-1c91afa63f4c service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:41:07.935748 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c1b8a-3b54-44ad-acff-1c91afa63f4c service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:41:07.936017 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15c1b8a-3b54-44ad-acff-1c91afa63f4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:41:07.937584 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-f15c1b8a-3b54-44ad-acff-1c91afa63f4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:41:08.046136 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15c1b8a-3b54-44ad-acff-1c91afa63f4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:41:08.047602 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15c1b8a-3b54-44ad-acff-1c91afa63f4c service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:41:08.064967 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 737/1461] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:41:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:41:10.258041 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f8324c-6da8-4a91-baa9-0387dce3ee66 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:41:10.258685 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f8324c-6da8-4a91-baa9-0387dce3ee66 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:41:10.258685 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f8324c-6da8-4a91-baa9-0387dce3ee66 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:41:10.260594 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 725/1462] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 10:41:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:41:10.265919 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782213f-34a6-40c7-974c-df5989b74202 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:41:10.266330 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782213f-34a6-40c7-974c-df5989b74202 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:41:10.266474 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782213f-34a6-40c7-974c-df5989b74202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:41:10.332993 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-4782213f-34a6-40c7-974c-df5989b74202 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:41:10.334188 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-4782213f-34a6-40c7-974c-df5989b74202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:41:10.372965 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:41:10.432137 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 738/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 10:41:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:41:10.719662 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f8324c-6da8-4a91-baa9-0387dce3ee66 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:41:10.721200 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-18f8324c-6da8-4a91-baa9-0387dce3ee66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:41:10.823856 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f8324c-6da8-4a91-baa9-0387dce3ee66 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:41:10.865397 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:41:10.903035 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3354604-7246-49d3-98c5-761613770627 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:41:10.903685 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3354604-7246-49d3-98c5-761613770627 service neutron] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:41:10.903967 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3354604-7246-49d3-98c5-761613770627 service neutron] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:41:10.904236 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3354604-7246-49d3-98c5-761613770627 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:41:10.905852 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-b3354604-7246-49d3-98c5-761613770627 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:41:11.013773 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3354604-7246-49d3-98c5-761613770627 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:41:11.015276 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3354604-7246-49d3-98c5-761613770627 service neutron] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:41:11.033431 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 726/1464] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:41:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:41:13.201676 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8dd12f-3de4-4c3d-8281-864091d8fa62 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:41:13.202258 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8dd12f-3de4-4c3d-8281-864091d8fa62 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:41:13.202258 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8dd12f-3de4-4c3d-8281-864091d8fa62 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:41:13.203612 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 739/1465] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 10:41:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:41:13.208865 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c57ddc-b19f-4fbb-9737-a575ffe57688 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:41:13.209315 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c57ddc-b19f-4fbb-9737-a575ffe57688 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:41:13.209422 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c57ddc-b19f-4fbb-9737-a575ffe57688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:41:13.276308 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-a5c57ddc-b19f-4fbb-9737-a575ffe57688 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:41:13.277511 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-a5c57ddc-b19f-4fbb-9737-a575ffe57688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:41:13.295665 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:41:13.378695 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 727/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 10:41:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:41:13.663140 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8dd12f-3de4-4c3d-8281-864091d8fa62 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:41:13.664692 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-8b8dd12f-3de4-4c3d-8281-864091d8fa62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:41:13.769109 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8dd12f-3de4-4c3d-8281-864091d8fa62 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:41:13.834016 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:41:13.841618 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf747214-a70d-4983-b89c-5dfccf5bf559 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:41:13.842231 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf747214-a70d-4983-b89c-5dfccf5bf559 service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:41:13.842510 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf747214-a70d-4983-b89c-5dfccf5bf559 service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:41:13.842784 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf747214-a70d-4983-b89c-5dfccf5bf559 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:41:13.844382 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-bf747214-a70d-4983-b89c-5dfccf5bf559 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:41:13.952506 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf747214-a70d-4983-b89c-5dfccf5bf559 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:41:13.953871 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf747214-a70d-4983-b89c-5dfccf5bf559 service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:41:13.972187 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 740/1467] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:41:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:41:16.143228 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7240f88a-1730-4118-acac-3693cc2e9fee None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:41:16.143836 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7240f88a-1730-4118-acac-3693cc2e9fee None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:41:16.143836 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7240f88a-1730-4118-acac-3693cc2e9fee None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:41:16.145094 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 728/1468] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 10:41:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:41:16.150269 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6120e8f1-a332-4ac6-848b-50719f34d2d2 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:41:16.151167 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6120e8f1-a332-4ac6-848b-50719f34d2d2 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:41:16.151665 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6120e8f1-a332-4ac6-848b-50719f34d2d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:41:16.217967 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-6120e8f1-a332-4ac6-848b-50719f34d2d2 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:41:16.219154 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-6120e8f1-a332-4ac6-848b-50719f34d2d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:41:16.268283 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:41:16.316920 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 741/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 10:41:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:41:16.599655 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7240f88a-1730-4118-acac-3693cc2e9fee None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:41:16.601202 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-7240f88a-1730-4118-acac-3693cc2e9fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:41:16.704173 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7240f88a-1730-4118-acac-3693cc2e9fee None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:41:16.752403 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:41:16.777946 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656d3033-d4b4-44bc-86fe-bc59d8a3ef66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:41:16.778482 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d3033-d4b4-44bc-86fe-bc59d8a3ef66 service neutron] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:41:16.778757 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d3033-d4b4-44bc-86fe-bc59d8a3ef66 service neutron] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:41:16.779024 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d3033-d4b4-44bc-86fe-bc59d8a3ef66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:41:16.780610 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-656d3033-d4b4-44bc-86fe-bc59d8a3ef66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:41:16.887669 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656d3033-d4b4-44bc-86fe-bc59d8a3ef66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:41:16.889113 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656d3033-d4b4-44bc-86fe-bc59d8a3ef66 service neutron] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:41:16.907038 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 729/1470] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:41:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:41:26.315364 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=22 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:41:26.904836 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=66 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:41:37.020016 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97158eb-48bf-4531-bc6a-9c7c00cd85fa None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:41:37.020586 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97158eb-48bf-4531-bc6a-9c7c00cd85fa None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:41:37.020586 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97158eb-48bf-4531-bc6a-9c7c00cd85fa None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:41:37.021884 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 742/1471] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:41:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:41:37.027653 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe8996a-d922-4f6c-b95b-028b2b7d0898 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:41:37.028084 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe8996a-d922-4f6c-b95b-028b2b7d0898 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:41:37.028237 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe8996a-d922-4f6c-b95b-028b2b7d0898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:41:37.094775 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-ffe8996a-d922-4f6c-b95b-028b2b7d0898 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:41:37.095921 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-ffe8996a-d922-4f6c-b95b-028b2b7d0898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:41:37.192087 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 730/1472] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:41:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:41:37.493489 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b707424-5ed5-4d76-a0ca-3c663e1c2779 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:41:37.494288 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b707424-5ed5-4d76-a0ca-3c663e1c2779 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:41:37.494711 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b707424-5ed5-4d76-a0ca-3c663e1c2779 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:41:37.496104 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 743/1473] 10.222.0.15 () {54 vars in 966 bytes} [Sun Sep 11 10:41:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:41:37.501121 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114de68a-d59a-447e-aecb-25be491d2abe None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:41:37.501436 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114de68a-d59a-447e-aecb-25be491d2abe None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:41:37.501738 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114de68a-d59a-447e-aecb-25be491d2abe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:41:37.578838 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-114de68a-d59a-447e-aecb-25be491d2abe None None] MFA Rules not processed for user `1632da5fbe384b5c849be1d4d86c0f1d`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:41:37.580064 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-114de68a-d59a-447e-aecb-25be491d2abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:41:37.600080 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:41:37.655496 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 731/1474] 10.222.0.15 () {58 vars in 1063 bytes} [Sun Sep 11 10:41:37 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:41:37.663084 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b707424-5ed5-4d76-a0ca-3c663e1c2779 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:41:37.664674 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-3b707424-5ed5-4d76-a0ca-3c663e1c2779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:41:37.768045 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b707424-5ed5-4d76-a0ca-3c663e1c2779 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:41:37.841377 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b5ccdc-5938-4181-ac95-299d7b5f824b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1632da5fbe384b5c849be1d4d86c0f1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:41:37.841985 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b5ccdc-5938-4181-ac95-299d7b5f824b service glance] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:41:37.842263 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b5ccdc-5938-4181-ac95-299d7b5f824b service glance] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:41:37.842530 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b5ccdc-5938-4181-ac95-299d7b5f824b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:41:37.844139 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-62b5ccdc-5938-4181-ac95-299d7b5f824b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:41:37.899805 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:41:37.954758 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b5ccdc-5938-4181-ac95-299d7b5f824b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:41:37.956251 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b5ccdc-5938-4181-ac95-299d7b5f824b service glance] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:41:37.974444 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 744/1475] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:41:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:41:40.221219 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7b62ee-5ade-4494-8b9c-0dcbfc869b4b None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:41:40.221766 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7b62ee-5ade-4494-8b9c-0dcbfc869b4b None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:41:40.221766 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7b62ee-5ade-4494-8b9c-0dcbfc869b4b None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:41:40.223098 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 732/1476] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:41:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:41:40.229266 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06261750-90c6-44d9-8f88-f4a4c1d82f9a None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:41:40.229962 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06261750-90c6-44d9-8f88-f4a4c1d82f9a None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:41:40.230093 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06261750-90c6-44d9-8f88-f4a4c1d82f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:41:40.296650 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-06261750-90c6-44d9-8f88-f4a4c1d82f9a None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:41:40.297847 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-06261750-90c6-44d9-8f88-f4a4c1d82f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:41:40.393984 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 745/1477] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:41:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:41:40.670629 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7b62ee-5ade-4494-8b9c-0dcbfc869b4b None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:41:40.672227 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-1b7b62ee-5ade-4494-8b9c-0dcbfc869b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:41:40.798118 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7b62ee-5ade-4494-8b9c-0dcbfc869b4b None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:41:40.850483 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:41:40.895688 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99de84dd-baf9-4a70-b6a1-2cf2e52614bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:41:40.896389 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99de84dd-baf9-4a70-b6a1-2cf2e52614bd admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:41:40.896665 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99de84dd-baf9-4a70-b6a1-2cf2e52614bd admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:41:40.896932 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99de84dd-baf9-4a70-b6a1-2cf2e52614bd admin admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:41:40.897560 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99de84dd-baf9-4a70-b6a1-2cf2e52614bd admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:41:40.899463 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: WARNING py.warnings [None req-99de84dd-baf9-4a70-b6a1-2cf2e52614bd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 11 10:41:40.899463 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: warnings.warn(msg) Sep 11 10:41:40.899463 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: Sep 11 10:41:40.901521 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99de84dd-baf9-4a70-b6a1-2cf2e52614bd admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:41:40.921155 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 733/1478] 10.222.0.15 () {56 vars in 1108 bytes} [Sun Sep 11 10:41:40 2022] GET /identity/v3/projects => generated 2553 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:41:43.104226 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ca0907-65fe-4df9-9bab-36daf86fbba6 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:41:43.105082 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ca0907-65fe-4df9-9bab-36daf86fbba6 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:41:43.105082 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ca0907-65fe-4df9-9bab-36daf86fbba6 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:41:43.106352 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 746/1479] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:41:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:41:43.111593 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f034ab6f-ac3a-4ba1-8222-db80f9aa3423 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:41:43.112027 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f034ab6f-ac3a-4ba1-8222-db80f9aa3423 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:41:43.112130 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f034ab6f-ac3a-4ba1-8222-db80f9aa3423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:41:43.185623 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-f034ab6f-ac3a-4ba1-8222-db80f9aa3423 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:41:43.186841 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-f034ab6f-ac3a-4ba1-8222-db80f9aa3423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:41:43.286093 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 734/1480] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:41:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 10:41:43.580680 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ca0907-65fe-4df9-9bab-36daf86fbba6 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:41:43.582305 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-d6ca0907-65fe-4df9-9bab-36daf86fbba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:41:43.592602 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:41:43.727356 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ca0907-65fe-4df9-9bab-36daf86fbba6 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:41:43.806562 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:41:43.840630 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9664b3-ef7e-41ce-9f05-c04afa404b11 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bd5356829b6454993bab9bad2f79fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:41:43.841161 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9664b3-ef7e-41ce-9f05-c04afa404b11 service nova] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:41:43.841464 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9664b3-ef7e-41ce-9f05-c04afa404b11 service nova] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:41:43.841734 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9664b3-ef7e-41ce-9f05-c04afa404b11 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:41:43.843334 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-fe9664b3-ef7e-41ce-9f05-c04afa404b11 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:41:43.937101 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9664b3-ef7e-41ce-9f05-c04afa404b11 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:41:43.938692 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9664b3-ef7e-41ce-9f05-c04afa404b11 service nova] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:41:43.956880 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 747/1481] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:41:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:41:46.301545 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b036db-3bfe-4164-9b18-75ab6da1f7a8 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:41:46.302125 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b036db-3bfe-4164-9b18-75ab6da1f7a8 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:41:46.302125 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b036db-3bfe-4164-9b18-75ab6da1f7a8 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:41:46.303407 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 735/1482] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:41:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:41:46.309315 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53409a-309e-48a4-8695-8dc29f428613 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:41:46.309755 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53409a-309e-48a4-8695-8dc29f428613 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:41:46.309849 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53409a-309e-48a4-8695-8dc29f428613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:41:46.376781 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-fb53409a-309e-48a4-8695-8dc29f428613 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:41:46.378020 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-fb53409a-309e-48a4-8695-8dc29f428613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:41:46.424709 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:41:46.473791 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 748/1483] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:41:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:41:46.762564 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b036db-3bfe-4164-9b18-75ab6da1f7a8 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:41:46.764125 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-32b036db-3bfe-4164-9b18-75ab6da1f7a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:41:46.787182 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:41:46.867492 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b036db-3bfe-4164-9b18-75ab6da1f7a8 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:41:46.941233 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09c40fc-78dd-427e-8166-b6cba8fa8861 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:41:46.941866 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09c40fc-78dd-427e-8166-b6cba8fa8861 service neutron] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:41:46.942144 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09c40fc-78dd-427e-8166-b6cba8fa8861 service neutron] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:41:46.942414 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09c40fc-78dd-427e-8166-b6cba8fa8861 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:41:46.944033 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-f09c40fc-78dd-427e-8166-b6cba8fa8861 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:41:47.017556 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:41:47.052038 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09c40fc-78dd-427e-8166-b6cba8fa8861 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:41:47.053452 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09c40fc-78dd-427e-8166-b6cba8fa8861 service neutron] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:41:47.072669 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 736/1484] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:41:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:41:49.281812 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54c8176-8a29-4517-8f4b-7002dbd4e8cc None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:41:49.282378 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54c8176-8a29-4517-8f4b-7002dbd4e8cc None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:41:49.282378 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54c8176-8a29-4517-8f4b-7002dbd4e8cc None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:41:49.283732 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 749/1485] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:41:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 10:41:49.289014 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7185b4d3-695f-4cc8-96bf-1a4bdc0ca503 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:41:49.289430 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7185b4d3-695f-4cc8-96bf-1a4bdc0ca503 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:41:49.289561 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7185b4d3-695f-4cc8-96bf-1a4bdc0ca503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:41:49.357926 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-7185b4d3-695f-4cc8-96bf-1a4bdc0ca503 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:41:49.359140 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-7185b4d3-695f-4cc8-96bf-1a4bdc0ca503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:41:49.456831 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 737/1486] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:41:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:41:49.750659 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54c8176-8a29-4517-8f4b-7002dbd4e8cc None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:41:49.752254 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-f54c8176-8a29-4517-8f4b-7002dbd4e8cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:41:49.855778 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54c8176-8a29-4517-8f4b-7002dbd4e8cc None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:41:49.927105 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:41:49.928618 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb541b5-69ad-44c5-be75-74440315aeb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:41:49.929292 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb541b5-69ad-44c5-be75-74440315aeb3 service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:41:49.929671 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb541b5-69ad-44c5-be75-74440315aeb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:41:49.929982 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb541b5-69ad-44c5-be75-74440315aeb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:41:49.932259 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-ffb541b5-69ad-44c5-be75-74440315aeb3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:41:50.041061 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb541b5-69ad-44c5-be75-74440315aeb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:41:50.042509 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb541b5-69ad-44c5-be75-74440315aeb3 service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:41:50.060805 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 750/1487] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:41:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:41:52.390217 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6f0b1f-2c64-40a0-ba10-08eb67685d0e None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:41:52.390778 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6f0b1f-2c64-40a0-ba10-08eb67685d0e None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:41:52.390778 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6f0b1f-2c64-40a0-ba10-08eb67685d0e None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:41:52.392118 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 738/1488] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:41:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:41:52.397267 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a40c83-40a0-4af2-a7dd-9217ea765eee None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:41:52.397647 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a40c83-40a0-4af2-a7dd-9217ea765eee None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:41:52.397822 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a40c83-40a0-4af2-a7dd-9217ea765eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:41:52.462827 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-a7a40c83-40a0-4af2-a7dd-9217ea765eee None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:41:52.464056 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-a7a40c83-40a0-4af2-a7dd-9217ea765eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:41:52.516088 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:41:52.562016 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 751/1489] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:41:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:41:52.869547 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6f0b1f-2c64-40a0-ba10-08eb67685d0e None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:41:52.871335 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-fb6f0b1f-2c64-40a0-ba10-08eb67685d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:41:52.909180 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:41:52.978602 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6f0b1f-2c64-40a0-ba10-08eb67685d0e None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:41:53.052208 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c0fd01-ba56-44c6-b8b4-67af2b080a7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:41:53.052814 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c0fd01-ba56-44c6-b8b4-67af2b080a7a service neutron] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:41:53.053090 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c0fd01-ba56-44c6-b8b4-67af2b080a7a service neutron] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:41:53.053360 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c0fd01-ba56-44c6-b8b4-67af2b080a7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:41:53.054934 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-19c0fd01-ba56-44c6-b8b4-67af2b080a7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:41:53.135770 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:41:53.164521 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c0fd01-ba56-44c6-b8b4-67af2b080a7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:41:53.165976 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c0fd01-ba56-44c6-b8b4-67af2b080a7a service neutron] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:41:53.184081 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 739/1490] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:41:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:41:55.544210 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4c4b4d-5d23-47f5-b8b2-78d75f624f99 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:41:55.544744 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4c4b4d-5d23-47f5-b8b2-78d75f624f99 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:41:55.544744 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4c4b4d-5d23-47f5-b8b2-78d75f624f99 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:41:55.546097 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 752/1491] 10.222.0.15 () {54 vars in 927 bytes} [Sun Sep 11 10:41:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:41:55.551529 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dca3e6-a4b8-42af-bc54-eb8d3e9cb91d None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:41:55.551914 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dca3e6-a4b8-42af-bc54-eb8d3e9cb91d None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:41:55.552075 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dca3e6-a4b8-42af-bc54-eb8d3e9cb91d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:41:55.617980 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-44dca3e6-a4b8-42af-bc54-eb8d3e9cb91d None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:41:55.619296 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-44dca3e6-a4b8-42af-bc54-eb8d3e9cb91d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:41:55.716946 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 740/1492] 10.222.0.15 () {58 vars in 1024 bytes} [Sun Sep 11 10:41:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:41:56.011646 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4c4b4d-5d23-47f5-b8b2-78d75f624f99 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:41:56.013179 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-6b4c4b4d-5d23-47f5-b8b2-78d75f624f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:41:56.113736 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4c4b4d-5d23-47f5-b8b2-78d75f624f99 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:41:56.187221 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208bf545-2682-463b-a8cf-4e6498c1c5da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:41:56.187861 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208bf545-2682-463b-a8cf-4e6498c1c5da service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:41:56.188145 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208bf545-2682-463b-a8cf-4e6498c1c5da service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:41:56.188423 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208bf545-2682-463b-a8cf-4e6498c1c5da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:41:56.189999 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-208bf545-2682-463b-a8cf-4e6498c1c5da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:41:56.200123 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:41:56.300983 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208bf545-2682-463b-a8cf-4e6498c1c5da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:41:56.302434 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208bf545-2682-463b-a8cf-4e6498c1c5da service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:41:56.321198 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 753/1493] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:41:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:42:05.715227 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=82 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:42:06.318808 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=50 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:43:09.140495 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c6528a-6418-4fde-8172-0ddfbf136147 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:43:09.141095 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c6528a-6418-4fde-8172-0ddfbf136147 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:43:09.141210 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c6528a-6418-4fde-8172-0ddfbf136147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:43:09.217815 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-03c6528a-6418-4fde-8172-0ddfbf136147 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:43:09.219019 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-03c6528a-6418-4fde-8172-0ddfbf136147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:43:09.314984 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 741/1494] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:43:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:43:09.323751 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bab2071-61d4-4976-a3ad-a3cf27fa27f3 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:43:09.324285 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bab2071-61d4-4976-a3ad-a3cf27fa27f3 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:43:09.324475 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bab2071-61d4-4976-a3ad-a3cf27fa27f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:43:09.403604 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-4bab2071-61d4-4976-a3ad-a3cf27fa27f3 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:43:09.404716 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-4bab2071-61d4-4976-a3ad-a3cf27fa27f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:43:09.498590 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 754/1495] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:43:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:43:09.503249 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c6528a-6418-4fde-8172-0ddfbf136147 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:43:09.504511 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-03c6528a-6418-4fde-8172-0ddfbf136147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:43:09.516400 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c6528a-6418-4fde-8172-0ddfbf136147 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:43:09.578685 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:43:09.612162 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba78ff45-9270-46f6-9ea9-ccfc846da50b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:43:09.612758 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba78ff45-9270-46f6-9ea9-ccfc846da50b admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:43:09.613034 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba78ff45-9270-46f6-9ea9-ccfc846da50b admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:43:09.613303 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba78ff45-9270-46f6-9ea9-ccfc846da50b admin admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:43:09.613926 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba78ff45-9270-46f6-9ea9-ccfc846da50b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:43:09.614221 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba78ff45-9270-46f6-9ea9-ccfc846da50b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:43:09.615213 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: WARNING py.warnings [None req-ba78ff45-9270-46f6-9ea9-ccfc846da50b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 11 10:43:09.615213 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: warnings.warn(msg) Sep 11 10:43:09.615213 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: Sep 11 10:43:09.616673 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba78ff45-9270-46f6-9ea9-ccfc846da50b admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:43:09.627558 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 742/1496] 10.222.0.15 () {58 vars in 1153 bytes} [Sun Sep 11 10:43:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:43:09.636379 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bab2071-61d4-4976-a3ad-a3cf27fa27f3 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:43:09.645503 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bab2071-61d4-4976-a3ad-a3cf27fa27f3 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:43:09.706114 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:43:09.739831 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e38e3f6-ef6a-47da-a0cc-452f99abcc81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:43:09.740435 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e38e3f6-ef6a-47da-a0cc-452f99abcc81 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:43:09.741263 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e38e3f6-ef6a-47da-a0cc-452f99abcc81 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:43:09.741536 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e38e3f6-ef6a-47da-a0cc-452f99abcc81 admin admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:43:09.742157 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e38e3f6-ef6a-47da-a0cc-452f99abcc81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:43:09.743904 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: WARNING py.warnings [None req-8e38e3f6-ef6a-47da-a0cc-452f99abcc81 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 11 10:43:09.743904 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: warnings.warn(msg) Sep 11 10:43:09.743904 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: Sep 11 10:43:09.745101 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e38e3f6-ef6a-47da-a0cc-452f99abcc81 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:43:09.800210 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 755/1497] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:43:09 2022] POST /identity/v3/projects => generated 379 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:43:09.804929 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba78ff45-9270-46f6-9ea9-ccfc846da50b admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:43:09.806436 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-ba78ff45-9270-46f6-9ea9-ccfc846da50b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:43:09.839455 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba78ff45-9270-46f6-9ea9-ccfc846da50b admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:43:09.933166 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279233e8-31da-4e89-926d-402fee2a3e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:43:09.933773 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279233e8-31da-4e89-926d-402fee2a3e3a admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:43:09.934054 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279233e8-31da-4e89-926d-402fee2a3e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:43:09.934327 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279233e8-31da-4e89-926d-402fee2a3e3a admin admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:43:09.934960 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279233e8-31da-4e89-926d-402fee2a3e3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:43:09.935995 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: WARNING py.warnings [None req-279233e8-31da-4e89-926d-402fee2a3e3a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 11 10:43:09.935995 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: warnings.warn(msg) Sep 11 10:43:09.935995 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: Sep 11 10:43:09.937205 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279233e8-31da-4e89-926d-402fee2a3e3a admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:43:09.980151 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 743/1498] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:43:09 2022] POST /identity/v3/users => generated 290 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:43:09.984389 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e38e3f6-ef6a-47da-a0cc-452f99abcc81 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:43:09.985885 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-8e38e3f6-ef6a-47da-a0cc-452f99abcc81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:43:10.047647 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e38e3f6-ef6a-47da-a0cc-452f99abcc81 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:43:10.129768 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:43:10.141543 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ee944f-4f6a-4b2c-b04d-2ded78aa486d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:43:10.142149 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee944f-4f6a-4b2c-b04d-2ded78aa486d admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:43:10.142423 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee944f-4f6a-4b2c-b04d-2ded78aa486d admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:43:10.142698 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ee944f-4f6a-4b2c-b04d-2ded78aa486d admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:43:10.143320 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ee944f-4f6a-4b2c-b04d-2ded78aa486d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:43:10.144359 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: WARNING py.warnings [None req-e0ee944f-4f6a-4b2c-b04d-2ded78aa486d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 11 10:43:10.144359 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: warnings.warn(msg) Sep 11 10:43:10.144359 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: Sep 11 10:43:10.145240 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ee944f-4f6a-4b2c-b04d-2ded78aa486d admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:43:10.157636 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 756/1499] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:43:09 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:43:10.161653 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279233e8-31da-4e89-926d-402fee2a3e3a admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:43:10.163279 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-279233e8-31da-4e89-926d-402fee2a3e3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:43:10.174580 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279233e8-31da-4e89-926d-402fee2a3e3a admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:43:10.200327 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:43:10.273162 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3ae920-e68d-4ac9-b6dc-eb576e60b1a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:43:10.273779 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3ae920-e68d-4ac9-b6dc-eb576e60b1a8 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:43:10.274055 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3ae920-e68d-4ac9-b6dc-eb576e60b1a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:43:10.274327 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3ae920-e68d-4ac9-b6dc-eb576e60b1a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:43:10.274962 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3ae920-e68d-4ac9-b6dc-eb576e60b1a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:43:10.276007 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: WARNING py.warnings [None req-7b3ae920-e68d-4ac9-b6dc-eb576e60b1a8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 11 10:43:10.276007 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: warnings.warn(msg) Sep 11 10:43:10.276007 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: Sep 11 10:43:10.277863 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3ae920-e68d-4ac9-b6dc-eb576e60b1a8 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:43:10.293549 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 744/1500] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:43:10 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:43:10.297743 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ee944f-4f6a-4b2c-b04d-2ded78aa486d admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:43:10.299270 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-e0ee944f-4f6a-4b2c-b04d-2ded78aa486d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:43:10.333814 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ee944f-4f6a-4b2c-b04d-2ded78aa486d admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:43:10.427247 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a82cc35-9ffe-41b5-b445-8a98788ed9bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:43:10.428706 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a82cc35-9ffe-41b5-b445-8a98788ed9bf admin admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:43:10.428995 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a82cc35-9ffe-41b5-b445-8a98788ed9bf admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:43:10.429279 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a82cc35-9ffe-41b5-b445-8a98788ed9bf admin admin] PATH_INFO: `/v3/projects/3745a695470b4d3cb715a2acf39160f6/users/2ceaecab5840415cbc3c4fcc735a2638/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:43:10.453348 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a82cc35-9ffe-41b5-b445-8a98788ed9bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=3745a695470b4d3cb715a2acf39160f6, user_id=2ceaecab5840415cbc3c4fcc735a2638, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:43:10.454382 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: WARNING py.warnings [None req-4a82cc35-9ffe-41b5-b445-8a98788ed9bf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 11 10:43:10.454382 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: warnings.warn(msg) Sep 11 10:43:10.454382 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: Sep 11 10:43:10.456379 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a82cc35-9ffe-41b5-b445-8a98788ed9bf admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:43:10.481431 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 757/1501] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:43:10 2022] PUT /identity/v3/projects/3745a695470b4d3cb715a2acf39160f6/users/2ceaecab5840415cbc3c4fcc735a2638/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:43:10.494542 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6c520-6d02-43f3-9513-a9081baec8d6 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:43:10.494838 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6c520-6d02-43f3-9513-a9081baec8d6 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:43:10.495127 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6c520-6d02-43f3-9513-a9081baec8d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:43:10.552621 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-b9d6c520-6d02-43f3-9513-a9081baec8d6 None None] MFA Rules not processed for user `2ceaecab5840415cbc3c4fcc735a2638`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:43:10.554556 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-b9d6c520-6d02-43f3-9513-a9081baec8d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:43:10.634418 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 745/1502] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:43:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:43:10.647982 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401da132-e9ba-4fa8-ac6c-23c841f7b6e8 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:43:10.648360 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401da132-e9ba-4fa8-ac6c-23c841f7b6e8 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:43:10.648643 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401da132-e9ba-4fa8-ac6c-23c841f7b6e8 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:43:10.649977 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 758/1503] 10.222.0.15 () {54 vars in 964 bytes} [Sun Sep 11 10:43:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:43:10.654664 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb563f7-b6e4-4dcb-9ce2-a45d43834e0a None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:43:10.654948 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb563f7-b6e4-4dcb-9ce2-a45d43834e0a None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:43:10.655219 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb563f7-b6e4-4dcb-9ce2-a45d43834e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:43:10.687607 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:43:10.735411 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-fbb563f7-b6e4-4dcb-9ce2-a45d43834e0a None None] MFA Rules not processed for user `0bbb481abb8e43788e3412ac7d908ad3`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:43:10.736655 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-fbb563f7-b6e4-4dcb-9ce2-a45d43834e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:43:10.812677 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 746/1504] 10.222.0.15 () {58 vars in 1061 bytes} [Sun Sep 11 10:43:10 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:43:10.822107 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401da132-e9ba-4fa8-ac6c-23c841f7b6e8 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:43:10.823645 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-401da132-e9ba-4fa8-ac6c-23c841f7b6e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:43:10.878203 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:43:10.933472 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401da132-e9ba-4fa8-ac6c-23c841f7b6e8 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:43:11.006570 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d72cad-779c-45fe-981f-375f9094e873 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:43:11.007187 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d72cad-779c-45fe-981f-375f9094e873 service cinder] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:43:11.007497 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d72cad-779c-45fe-981f-375f9094e873 service cinder] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:43:11.007773 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d72cad-779c-45fe-981f-375f9094e873 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:43:11.009345 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-08d72cad-779c-45fe-981f-375f9094e873 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:43:11.106376 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d72cad-779c-45fe-981f-375f9094e873 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:43:11.107782 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d72cad-779c-45fe-981f-375f9094e873 service cinder] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:43:11.111184 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:43:11.126049 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 759/1505] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:43:10 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:43:11.171790 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e94120-66df-449a-9b87-cf74255b3142 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:43:11.172210 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e94120-66df-449a-9b87-cf74255b3142 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:43:11.172339 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e94120-66df-449a-9b87-cf74255b3142 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:43:11.173583 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 747/1506] 10.222.0.15 () {52 vars in 830 bytes} [Sun Sep 11 10:43:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:43:11.177223 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d72cad-779c-45fe-981f-375f9094e873 service cinder] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:43:11.178600 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-08d72cad-779c-45fe-981f-375f9094e873 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:43:11.277281 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d72cad-779c-45fe-981f-375f9094e873 service cinder] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:43:11.370810 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa97a2f-21c9-44e2-8cef-82d9ebff17d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:43:11.371488 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa97a2f-21c9-44e2-8cef-82d9ebff17d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:43:11.371663 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa97a2f-21c9-44e2-8cef-82d9ebff17d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:43:11.371935 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa97a2f-21c9-44e2-8cef-82d9ebff17d2 admin admin] PATH_INFO: `/v3/users/2ceaecab5840415cbc3c4fcc735a2638` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:43:11.385077 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa97a2f-21c9-44e2-8cef-82d9ebff17d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ceaecab5840415cbc3c4fcc735a2638)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:43:11.386101 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: WARNING py.warnings [None req-baa97a2f-21c9-44e2-8cef-82d9ebff17d2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 11 10:43:11.386101 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: warnings.warn(msg) Sep 11 10:43:11.386101 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: Sep 11 10:43:11.387318 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa97a2f-21c9-44e2-8cef-82d9ebff17d2 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:43:11.397514 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:43:11.454101 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-baa97a2f-21c9-44e2-8cef-82d9ebff17d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ceaecab5840415cbc3c4fcc735a2638'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:43:11.464291 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-baa97a2f-21c9-44e2-8cef-82d9ebff17d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ceaecab5840415cbc3c4fcc735a2638'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:43:11.464968 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-baa97a2f-21c9-44e2-8cef-82d9ebff17d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ceaecab5840415cbc3c4fcc735a2638'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:43:11.465400 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-baa97a2f-21c9-44e2-8cef-82d9ebff17d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ceaecab5840415cbc3c4fcc735a2638'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:43:11.488915 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-baa97a2f-21c9-44e2-8cef-82d9ebff17d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ceaecab5840415cbc3c4fcc735a2638'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:43:11.508179 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 760/1507] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:43:11 2022] DELETE /identity/v3/users/2ceaecab5840415cbc3c4fcc735a2638 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:43:11.512754 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a542b5f2-7db5-435a-be8b-b32c5ef25d0b None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:43:11.513041 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a542b5f2-7db5-435a-be8b-b32c5ef25d0b None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:43:11.513313 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a542b5f2-7db5-435a-be8b-b32c5ef25d0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:43:11.570520 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-a542b5f2-7db5-435a-be8b-b32c5ef25d0b None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:43:11.571754 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-a542b5f2-7db5-435a-be8b-b32c5ef25d0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:43:11.647109 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:43:11.659380 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 748/1508] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:43:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:43:11.669108 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa97a2f-21c9-44e2-8cef-82d9ebff17d2 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:43:11.670317 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-baa97a2f-21c9-44e2-8cef-82d9ebff17d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:43:11.771791 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa97a2f-21c9-44e2-8cef-82d9ebff17d2 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:43:11.826550 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:43:11.847563 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556f61cb-aa4f-4831-9601-4f3f005d1571 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:43:11.848184 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556f61cb-aa4f-4831-9601-4f3f005d1571 service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:43:11.848463 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556f61cb-aa4f-4831-9601-4f3f005d1571 service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:43:11.848735 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556f61cb-aa4f-4831-9601-4f3f005d1571 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:43:11.850348 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-556f61cb-aa4f-4831-9601-4f3f005d1571 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:43:11.958671 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556f61cb-aa4f-4831-9601-4f3f005d1571 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:43:11.960094 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556f61cb-aa4f-4831-9601-4f3f005d1571 service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:43:11.978615 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 761/1509] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:43:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:43:12.299070 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a542b5f2-7db5-435a-be8b-b32c5ef25d0b None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:43:12.300383 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-a542b5f2-7db5-435a-be8b-b32c5ef25d0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:43:12.312206 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a542b5f2-7db5-435a-be8b-b32c5ef25d0b None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:43:12.413934 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d62b362-f43a-4758-a039-550164f3324a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:43:12.414565 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d62b362-f43a-4758-a039-550164f3324a admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:43:12.414842 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d62b362-f43a-4758-a039-550164f3324a admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:43:12.415109 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d62b362-f43a-4758-a039-550164f3324a admin admin] PATH_INFO: `/v3/projects/3745a695470b4d3cb715a2acf39160f6` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:43:12.425082 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d62b362-f43a-4758-a039-550164f3324a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3745a695470b4d3cb715a2acf39160f6)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:43:12.426099 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: WARNING py.warnings [None req-8d62b362-f43a-4758-a039-550164f3324a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 11 10:43:12.426099 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: warnings.warn(msg) Sep 11 10:43:12.426099 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: Sep 11 10:43:12.427323 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d62b362-f43a-4758-a039-550164f3324a admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:43:12.485382 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-8d62b362-f43a-4758-a039-550164f3324a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3745a695470b4d3cb715a2acf39160f6', 'request_id': 'req-8d62b362-f43a-4758-a039-550164f3324a'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:43:12.498696 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-8d62b362-f43a-4758-a039-550164f3324a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3745a695470b4d3cb715a2acf39160f6', 'request_id': 'req-8d62b362-f43a-4758-a039-550164f3324a'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:43:12.508610 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-8d62b362-f43a-4758-a039-550164f3324a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3745a695470b4d3cb715a2acf39160f6', 'request_id': 'req-8d62b362-f43a-4758-a039-550164f3324a'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:43:12.531304 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-8d62b362-f43a-4758-a039-550164f3324a admin admin] The token cache is being invalidate because project 3745a695470b4d3cb715a2acf39160f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90653) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:43:12.531655 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-8d62b362-f43a-4758-a039-550164f3324a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:43:12.532180 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-8d62b362-f43a-4758-a039-550164f3324a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:43:12.533482 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 749/1510] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:43:12 2022] DELETE /identity/v3/projects/3745a695470b4d3cb715a2acf39160f6 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:43:21.976353 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=62 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:43:22.529351 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:44:37.737924 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4d2c7d-cfc5-4f3f-8243-3d6298ac6964 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:44:37.738488 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4d2c7d-cfc5-4f3f-8243-3d6298ac6964 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:44:37.738488 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4d2c7d-cfc5-4f3f-8243-3d6298ac6964 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:44:37.739770 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 762/1511] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 10:44:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:44:37.744871 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a73cdc1-0b2c-4f7f-a062-5498640550d9 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:44:37.745308 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a73cdc1-0b2c-4f7f-a062-5498640550d9 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:44:37.745412 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a73cdc1-0b2c-4f7f-a062-5498640550d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:44:37.826430 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-2a73cdc1-0b2c-4f7f-a062-5498640550d9 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:44:37.827667 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-2a73cdc1-0b2c-4f7f-a062-5498640550d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:44:37.934173 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 750/1512] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 10:44:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:44:38.088330 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4d2c7d-cfc5-4f3f-8243-3d6298ac6964 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:44:38.090419 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-cc4d2c7d-cfc5-4f3f-8243-3d6298ac6964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:44:38.210090 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4d2c7d-cfc5-4f3f-8243-3d6298ac6964 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:44:38.215504 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b22fb-49da-4bee-8b5d-16c0f547d4c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bd5356829b6454993bab9bad2f79fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:44:38.216113 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b22fb-49da-4bee-8b5d-16c0f547d4c6 service nova] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:44:38.216394 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b22fb-49da-4bee-8b5d-16c0f547d4c6 service nova] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:44:38.216663 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b22fb-49da-4bee-8b5d-16c0f547d4c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:44:38.224612 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6b22fb-49da-4bee-8b5d-16c0f547d4c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:44:38.226765 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6b22fb-49da-4bee-8b5d-16c0f547d4c6 service nova] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:44:38.236361 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 763/1513] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:44:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:44:39.332353 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3671ede-7494-4db7-a8df-d9e8c1388525 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:44:39.332863 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3671ede-7494-4db7-a8df-d9e8c1388525 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:44:39.332949 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3671ede-7494-4db7-a8df-d9e8c1388525 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:44:39.334895 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 751/1514] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 10:44:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:44:39.341514 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa319e-934b-4616-8135-d624bc1d908d None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:44:39.342017 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa319e-934b-4616-8135-d624bc1d908d None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:44:39.342114 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa319e-934b-4616-8135-d624bc1d908d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:44:39.362978 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-bdfa319e-934b-4616-8135-d624bc1d908d None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:44:39.364251 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-bdfa319e-934b-4616-8135-d624bc1d908d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:44:39.372447 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 764/1515] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 10:44:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:44:39.524825 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3671ede-7494-4db7-a8df-d9e8c1388525 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:44:39.539539 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3671ede-7494-4db7-a8df-d9e8c1388525 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:44:39.544839 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3196c016-9665-43e3-90e1-484fd72cd223 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bd5356829b6454993bab9bad2f79fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:44:39.545427 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3196c016-9665-43e3-90e1-484fd72cd223 service nova] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:44:39.545705 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3196c016-9665-43e3-90e1-484fd72cd223 service nova] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:44:39.545972 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3196c016-9665-43e3-90e1-484fd72cd223 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:44:39.553998 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3196c016-9665-43e3-90e1-484fd72cd223 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:44:39.555618 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3196c016-9665-43e3-90e1-484fd72cd223 service nova] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:44:39.564776 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 752/1516] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:44:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:44:39.769594 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb877b3-b3a1-4e31-a9b1-bc84dec16946 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:44:39.769881 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb877b3-b3a1-4e31-a9b1-bc84dec16946 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:44:39.770158 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb877b3-b3a1-4e31-a9b1-bc84dec16946 None None] PATH_INFO: `/v3` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:44:39.771530 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 765/1517] 10.222.0.25 () {54 vars in 925 bytes} [Sun Sep 11 10:44:39 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:44:39.777953 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7923921-5378-4917-903e-88738408b3dc None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:44:39.778272 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7923921-5378-4917-903e-88738408b3dc None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:44:39.778564 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7923921-5378-4917-903e-88738408b3dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:44:39.839700 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-b7923921-5378-4917-903e-88738408b3dc None None] MFA Rules not processed for user `08477b4c88f04c9e97391aba401879e0`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:44:39.840826 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-b7923921-5378-4917-903e-88738408b3dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:44:39.858779 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:44:39.914246 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 753/1518] 10.222.0.25 () {58 vars in 1015 bytes} [Sun Sep 11 10:44:39 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:44:48.842856 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da70e12a-4935-4fa1-ac15-eb7764d1e23f None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:44:48.843337 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da70e12a-4935-4fa1-ac15-eb7764d1e23f None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:44:48.843492 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da70e12a-4935-4fa1-ac15-eb7764d1e23f None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:44:48.844785 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 766/1519] 10.222.0.15 () {54 vars in 961 bytes} [Sun Sep 11 10:44:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:44:48.850404 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ca757-1217-48ab-b7da-1c991aed5f28 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:44:48.850842 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ca757-1217-48ab-b7da-1c991aed5f28 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:44:48.851037 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ca757-1217-48ab-b7da-1c991aed5f28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:44:48.871362 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-0f7ca757-1217-48ab-b7da-1c991aed5f28 None None] MFA Rules not processed for user `08477b4c88f04c9e97391aba401879e0`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:44:48.872611 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-0f7ca757-1217-48ab-b7da-1c991aed5f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:44:48.882759 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 754/1520] 10.222.0.15 () {58 vars in 1058 bytes} [Sun Sep 11 10:44:48 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:44:48.892040 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da70e12a-4935-4fa1-ac15-eb7764d1e23f None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:44:48.905114 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da70e12a-4935-4fa1-ac15-eb7764d1e23f None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:44:48.909989 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e89394-6ee6-46e8-b6c5-0dfceec1c344 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08477b4c88f04c9e97391aba401879e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:44:48.910582 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e89394-6ee6-46e8-b6c5-0dfceec1c344 service placement] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:44:48.910863 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e89394-6ee6-46e8-b6c5-0dfceec1c344 service placement] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:44:48.911136 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e89394-6ee6-46e8-b6c5-0dfceec1c344 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:44:48.919765 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e89394-6ee6-46e8-b6c5-0dfceec1c344 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:44:48.921361 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e89394-6ee6-46e8-b6c5-0dfceec1c344 service placement] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:44:48.931622 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 767/1521] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:44:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:44:51.040902 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5bd4c6-2e01-4604-bd26-689c25260255 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:44:51.041456 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5bd4c6-2e01-4604-bd26-689c25260255 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:44:51.041456 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5bd4c6-2e01-4604-bd26-689c25260255 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:44:51.042777 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 755/1522] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 10:44:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:44:51.048951 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5034feef-d205-464e-bd09-8d0fb89c5094 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:44:51.049466 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5034feef-d205-464e-bd09-8d0fb89c5094 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:44:51.049576 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5034feef-d205-464e-bd09-8d0fb89c5094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:44:51.069605 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-5034feef-d205-464e-bd09-8d0fb89c5094 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:44:51.070913 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-5034feef-d205-464e-bd09-8d0fb89c5094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:44:51.078655 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 768/1523] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 10:44:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:44:51.234309 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5bd4c6-2e01-4604-bd26-689c25260255 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:44:51.249535 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5bd4c6-2e01-4604-bd26-689c25260255 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:44:51.254862 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c599af-687a-4644-9d65-ef77fde57134 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bd5356829b6454993bab9bad2f79fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:44:51.255469 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c599af-687a-4644-9d65-ef77fde57134 service nova] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:44:51.255749 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c599af-687a-4644-9d65-ef77fde57134 service nova] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:44:51.256022 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c599af-687a-4644-9d65-ef77fde57134 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:44:51.264570 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c599af-687a-4644-9d65-ef77fde57134 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:44:51.266161 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c599af-687a-4644-9d65-ef77fde57134 service nova] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:44:51.276241 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 756/1524] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:44:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:44:52.349522 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf3b27-0ce9-469e-9ecf-70badfdcbbf3 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:44:52.350011 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf3b27-0ce9-469e-9ecf-70badfdcbbf3 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:44:52.350119 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf3b27-0ce9-469e-9ecf-70badfdcbbf3 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:44:52.351436 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 769/1525] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 10:44:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:44:52.357922 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b552e0-a8ec-4ca8-98f5-34f83e25ca51 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:44:52.358413 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b552e0-a8ec-4ca8-98f5-34f83e25ca51 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:44:52.358517 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b552e0-a8ec-4ca8-98f5-34f83e25ca51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:44:52.378139 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-85b552e0-a8ec-4ca8-98f5-34f83e25ca51 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:44:52.379332 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-85b552e0-a8ec-4ca8-98f5-34f83e25ca51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:44:52.387803 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 757/1526] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 10:44:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 10:44:52.538649 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bf3b27-0ce9-469e-9ecf-70badfdcbbf3 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:44:52.553060 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bf3b27-0ce9-469e-9ecf-70badfdcbbf3 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:44:52.559330 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca56a10d-b5de-47bf-bf0c-464a70575665 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bd5356829b6454993bab9bad2f79fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:44:52.559954 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca56a10d-b5de-47bf-bf0c-464a70575665 service nova] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:44:52.560233 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca56a10d-b5de-47bf-bf0c-464a70575665 service nova] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:44:52.560506 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca56a10d-b5de-47bf-bf0c-464a70575665 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:44:52.568456 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca56a10d-b5de-47bf-bf0c-464a70575665 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:44:52.570399 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca56a10d-b5de-47bf-bf0c-464a70575665 service nova] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:44:52.579627 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 770/1527] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:44:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:44:56.608745 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64df251-d981-4d4d-bee8-f22e57f7dd92 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:44:56.609553 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64df251-d981-4d4d-bee8-f22e57f7dd92 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:44:56.609553 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64df251-d981-4d4d-bee8-f22e57f7dd92 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:44:56.611067 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 758/1528] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 10:44:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:44:56.616576 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5845657d-7f61-4159-acc5-15f3d9b04b52 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:44:56.617004 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5845657d-7f61-4159-acc5-15f3d9b04b52 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:44:56.617193 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5845657d-7f61-4159-acc5-15f3d9b04b52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:44:56.636259 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-5845657d-7f61-4159-acc5-15f3d9b04b52 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:44:56.637446 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-5845657d-7f61-4159-acc5-15f3d9b04b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:44:56.645219 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 771/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 10:44:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:44:56.935802 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64df251-d981-4d4d-bee8-f22e57f7dd92 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:44:56.937472 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-c64df251-d981-4d4d-bee8-f22e57f7dd92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:44:57.014669 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64df251-d981-4d4d-bee8-f22e57f7dd92 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:44:57.018130 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83538c4b-4eb1-4336-8480-4606e268f4af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:44:57.018726 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83538c4b-4eb1-4336-8480-4606e268f4af service neutron] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:44:57.019005 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83538c4b-4eb1-4336-8480-4606e268f4af service neutron] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:44:57.019281 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83538c4b-4eb1-4336-8480-4606e268f4af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:44:57.028504 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83538c4b-4eb1-4336-8480-4606e268f4af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:44:57.029915 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83538c4b-4eb1-4336-8480-4606e268f4af service neutron] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:44:57.040343 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 759/1530] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:44:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:44:59.197387 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb32ad-e278-4f9b-8b18-06d221e150e5 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:44:59.197989 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb32ad-e278-4f9b-8b18-06d221e150e5 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:44:59.197989 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb32ad-e278-4f9b-8b18-06d221e150e5 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:44:59.199336 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 772/1531] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 10:44:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:44:59.204626 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f15de-d527-4d3a-911d-e880fb29095e None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:44:59.205056 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f15de-d527-4d3a-911d-e880fb29095e None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:44:59.205159 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f15de-d527-4d3a-911d-e880fb29095e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:44:59.225383 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-fd1f15de-d527-4d3a-911d-e880fb29095e None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:44:59.226791 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-fd1f15de-d527-4d3a-911d-e880fb29095e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:44:59.234717 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 760/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 10:44:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:44:59.539796 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeb32ad-e278-4f9b-8b18-06d221e150e5 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:44:59.550882 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeb32ad-e278-4f9b-8b18-06d221e150e5 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:44:59.555556 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b19477a-37f1-4496-81a7-5d3ebaf682a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:44:59.556158 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b19477a-37f1-4496-81a7-5d3ebaf682a0 service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:44:59.556453 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b19477a-37f1-4496-81a7-5d3ebaf682a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:44:59.556858 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b19477a-37f1-4496-81a7-5d3ebaf682a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:44:59.565812 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b19477a-37f1-4496-81a7-5d3ebaf682a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:44:59.567219 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b19477a-37f1-4496-81a7-5d3ebaf682a0 service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:44:59.577495 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 773/1533] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:44:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:45:03.186763 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b3eebc-5a03-4928-91d0-c96bc3bed4e9 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:45:03.187306 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b3eebc-5a03-4928-91d0-c96bc3bed4e9 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:45:03.187306 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b3eebc-5a03-4928-91d0-c96bc3bed4e9 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:45:03.188643 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 761/1534] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 10:45:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:45:03.193915 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0803cb-def6-4118-984e-89a2f37301b9 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:45:03.194338 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0803cb-def6-4118-984e-89a2f37301b9 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:45:03.194457 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0803cb-def6-4118-984e-89a2f37301b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:45:03.202321 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=81 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:45:03.215182 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-7a0803cb-def6-4118-984e-89a2f37301b9 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:45:03.216391 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-7a0803cb-def6-4118-984e-89a2f37301b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:45:03.224023 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 774/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 10:45:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 10:45:03.529600 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b3eebc-5a03-4928-91d0-c96bc3bed4e9 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:45:03.531270 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-23b3eebc-5a03-4928-91d0-c96bc3bed4e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:45:03.596234 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:45:03.609790 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b3eebc-5a03-4928-91d0-c96bc3bed4e9 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:45:03.613314 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffad73f9-aaa4-411a-889c-eed8f424386d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:45:03.613918 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffad73f9-aaa4-411a-889c-eed8f424386d service cinder] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:45:03.614196 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffad73f9-aaa4-411a-889c-eed8f424386d service cinder] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:45:03.614467 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffad73f9-aaa4-411a-889c-eed8f424386d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:45:03.623849 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffad73f9-aaa4-411a-889c-eed8f424386d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:45:03.625260 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffad73f9-aaa4-411a-889c-eed8f424386d service cinder] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:45:03.636017 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 762/1536] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:45:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:45:05.795511 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4008797e-e164-44c9-92c1-d1b76fb49881 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:45:05.796093 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4008797e-e164-44c9-92c1-d1b76fb49881 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:45:05.796093 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4008797e-e164-44c9-92c1-d1b76fb49881 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:45:05.797343 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 775/1537] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 10:45:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:45:05.802661 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bb6d9f-14f2-40b8-9f56-41a29185d063 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:45:05.803120 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bb6d9f-14f2-40b8-9f56-41a29185d063 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:45:05.803376 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bb6d9f-14f2-40b8-9f56-41a29185d063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:45:05.823056 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-01bb6d9f-14f2-40b8-9f56-41a29185d063 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:45:05.824265 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-01bb6d9f-14f2-40b8-9f56-41a29185d063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:45:05.831891 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 763/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 10:45:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:45:06.135165 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4008797e-e164-44c9-92c1-d1b76fb49881 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:45:06.136876 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-4008797e-e164-44c9-92c1-d1b76fb49881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:45:06.149803 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4008797e-e164-44c9-92c1-d1b76fb49881 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:45:06.153335 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f373b0f-4da3-4b84-8d5c-0839f761f444 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:45:06.154041 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f373b0f-4da3-4b84-8d5c-0839f761f444 service cinder] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:45:06.154331 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f373b0f-4da3-4b84-8d5c-0839f761f444 service cinder] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:45:06.154607 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f373b0f-4da3-4b84-8d5c-0839f761f444 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:45:06.164001 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f373b0f-4da3-4b84-8d5c-0839f761f444 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:45:06.165405 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f373b0f-4da3-4b84-8d5c-0839f761f444 service cinder] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:45:06.174592 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 776/1539] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:45:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:45:15.818024 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=11 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:45:16.170587 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=8 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:46:31.585523 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c1865-4553-4d71-97f0-b87db6214e1f None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:31.586130 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c1865-4553-4d71-97f0-b87db6214e1f None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:31.586130 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c1865-4553-4d71-97f0-b87db6214e1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:31.612116 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-e07c1865-4553-4d71-97f0-b87db6214e1f None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:46:31.613704 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-e07c1865-4553-4d71-97f0-b87db6214e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:31.627869 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 764/1540] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:46:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:46:31.637905 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497f4282-dab4-47be-812c-b637ae88df6f None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:31.639752 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497f4282-dab4-47be-812c-b637ae88df6f None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:31.639752 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497f4282-dab4-47be-812c-b637ae88df6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:31.665084 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-497f4282-dab4-47be-812c-b637ae88df6f None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:46:31.666339 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-497f4282-dab4-47be-812c-b637ae88df6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:31.672566 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6b566d-8194-4c73-a456-4ee3e674a4e1 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:31.672912 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6b566d-8194-4c73-a456-4ee3e674a4e1 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:31.673119 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6b566d-8194-4c73-a456-4ee3e674a4e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:31.699019 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 777/1541] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:46:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:46:31.704405 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497f4282-dab4-47be-812c-b637ae88df6f None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:31.715265 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-dd6b566d-8194-4c73-a456-4ee3e674a4e1 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:46:31.716514 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-dd6b566d-8194-4c73-a456-4ee3e674a4e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:31.727005 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497f4282-dab4-47be-812c-b637ae88df6f None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:31.737342 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 765/1542] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:46:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:46:31.738424 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570050c5-4808-4cdb-8a93-f096d34847a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:31.739398 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570050c5-4808-4cdb-8a93-f096d34847a2 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:31.740851 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a936d-cef2-4af5-ab6e-68814b415555 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:31.741314 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570050c5-4808-4cdb-8a93-f096d34847a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:31.741751 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570050c5-4808-4cdb-8a93-f096d34847a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:31.742565 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570050c5-4808-4cdb-8a93-f096d34847a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:46:31.743002 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a936d-cef2-4af5-ab6e-68814b415555 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:31.743449 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a936d-cef2-4af5-ab6e-68814b415555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:31.744090 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570050c5-4808-4cdb-8a93-f096d34847a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:31.747996 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: WARNING py.warnings [None req-570050c5-4808-4cdb-8a93-f096d34847a2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 11 10:46:31.747996 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: warnings.warn(msg) Sep 11 10:46:31.747996 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: Sep 11 10:46:31.749435 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570050c5-4808-4cdb-8a93-f096d34847a2 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:31.779647 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 778/1543] 10.222.0.15 () {58 vars in 1153 bytes} [Sun Sep 11 10:46:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:46:31.781331 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dd1c81-e13e-4fa1-81b6-583e9982ae93 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:31.784185 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dd1c81-e13e-4fa1-81b6-583e9982ae93 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:31.784666 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dd1c81-e13e-4fa1-81b6-583e9982ae93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:31.791958 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-c69a936d-cef2-4af5-ab6e-68814b415555 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:46:31.796945 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-c69a936d-cef2-4af5-ab6e-68814b415555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:31.815343 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-c5dd1c81-e13e-4fa1-81b6-583e9982ae93 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:46:31.817110 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-c5dd1c81-e13e-4fa1-81b6-583e9982ae93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:31.820456 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 766/1544] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:46:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:46:31.824296 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594f2f4d-5156-4d21-8b01-28cbe2210742 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:31.826177 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594f2f4d-5156-4d21-8b01-28cbe2210742 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:31.826686 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594f2f4d-5156-4d21-8b01-28cbe2210742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:31.833585 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 779/1545] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:46:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:46:31.834672 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dd1c81-e13e-4fa1-81b6-583e9982ae93 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:31.857696 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dd1c81-e13e-4fa1-81b6-583e9982ae93 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:31.868504 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-594f2f4d-5156-4d21-8b01-28cbe2210742 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:46:31.870150 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04983b3-19de-4072-82ce-17c52355d6af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:31.871037 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04983b3-19de-4072-82ce-17c52355d6af admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:31.871566 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04983b3-19de-4072-82ce-17c52355d6af admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:31.872059 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04983b3-19de-4072-82ce-17c52355d6af admin admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:31.873090 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04983b3-19de-4072-82ce-17c52355d6af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:31.875226 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04983b3-19de-4072-82ce-17c52355d6af admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:31.876821 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-594f2f4d-5156-4d21-8b01-28cbe2210742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:31.891609 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 767/1546] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:46:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:46:31.893431 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1960a5f2-e4f2-46f5-b7c7-a7525f11aee6 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:31.894625 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1960a5f2-e4f2-46f5-b7c7-a7525f11aee6 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:31.895296 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1960a5f2-e4f2-46f5-b7c7-a7525f11aee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:31.916365 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-1960a5f2-e4f2-46f5-b7c7-a7525f11aee6 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:46:31.917567 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-1960a5f2-e4f2-46f5-b7c7-a7525f11aee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:31.926695 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 768/1547] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:46:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:46:31.927959 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564dff88-5082-4cbf-94ee-074457449782 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:31.928223 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564dff88-5082-4cbf-94ee-074457449782 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:31.928498 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564dff88-5082-4cbf-94ee-074457449782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:31.933214 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 780/1548] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:46:31 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:46:31.934949 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04983b3-19de-4072-82ce-17c52355d6af admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:31.949102 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-564dff88-5082-4cbf-94ee-074457449782 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:46:31.950306 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-564dff88-5082-4cbf-94ee-074457449782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:32.000353 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04983b3-19de-4072-82ce-17c52355d6af admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:32.005101 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ed9747-ef51-49c1-8832-1fcb8be3bea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:32.005718 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed9747-ef51-49c1-8832-1fcb8be3bea3 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:32.005990 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed9747-ef51-49c1-8832-1fcb8be3bea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:32.006282 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed9747-ef51-49c1-8832-1fcb8be3bea3 admin admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:32.006903 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ed9747-ef51-49c1-8832-1fcb8be3bea3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:46:32.007206 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ed9747-ef51-49c1-8832-1fcb8be3bea3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:32.008852 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ed9747-ef51-49c1-8832-1fcb8be3bea3 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:32.013853 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 769/1549] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:46:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:46:32.015066 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde57bd-9c0b-4a58-bd66-ab3377fa0d2d None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:32.015353 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde57bd-9c0b-4a58-bd66-ab3377fa0d2d None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:32.015654 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde57bd-9c0b-4a58-bd66-ab3377fa0d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:32.023374 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 781/1550] 10.222.0.15 () {58 vars in 1153 bytes} [Sun Sep 11 10:46:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:46:32.024632 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a43afe0-1e48-4c69-9a45-7aa6a1128c6d None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:32.024920 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a43afe0-1e48-4c69-9a45-7aa6a1128c6d None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:32.025244 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a43afe0-1e48-4c69-9a45-7aa6a1128c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:32.036329 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-1fde57bd-9c0b-4a58-bd66-ab3377fa0d2d None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:46:32.037822 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-1fde57bd-9c0b-4a58-bd66-ab3377fa0d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:32.045570 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-8a43afe0-1e48-4c69-9a45-7aa6a1128c6d None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:46:32.045903 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 770/1551] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:46:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:46:32.046755 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-8a43afe0-1e48-4c69-9a45-7aa6a1128c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:32.047128 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb71b21-807b-4a74-ab64-ce596ec08c7c None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:32.047412 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb71b21-807b-4a74-ab64-ce596ec08c7c None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:32.047727 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb71b21-807b-4a74-ab64-ce596ec08c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:32.055568 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 782/1552] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:46:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:46:32.056844 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f82c3-e138-4a39-8ccb-bc75b1062833 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:32.057137 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f82c3-e138-4a39-8ccb-bc75b1062833 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:32.057417 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f82c3-e138-4a39-8ccb-bc75b1062833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:32.068785 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-fdb71b21-807b-4a74-ab64-ce596ec08c7c None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:46:32.070861 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-fdb71b21-807b-4a74-ab64-ce596ec08c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:32.077247 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-188f82c3-e138-4a39-8ccb-bc75b1062833 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:46:32.078420 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-188f82c3-e138-4a39-8ccb-bc75b1062833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:32.079534 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 771/1553] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:46:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:46:32.080718 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f135761-dc33-4794-8f8c-69acb094fb98 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:32.081002 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f135761-dc33-4794-8f8c-69acb094fb98 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:32.081268 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f135761-dc33-4794-8f8c-69acb094fb98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:32.087281 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 783/1554] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:46:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:46:32.088021 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188f82c3-e138-4a39-8ccb-bc75b1062833 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:32.099461 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188f82c3-e138-4a39-8ccb-bc75b1062833 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:32.101502 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-1f135761-dc33-4794-8f8c-69acb094fb98 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:46:32.102693 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-1f135761-dc33-4794-8f8c-69acb094fb98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:32.104180 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326d5364-4f23-4eb2-86f4-76a6cbadec82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:32.104800 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326d5364-4f23-4eb2-86f4-76a6cbadec82 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:32.105079 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326d5364-4f23-4eb2-86f4-76a6cbadec82 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:32.105349 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326d5364-4f23-4eb2-86f4-76a6cbadec82 admin admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:32.105968 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326d5364-4f23-4eb2-86f4-76a6cbadec82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:32.107310 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: WARNING py.warnings [None req-326d5364-4f23-4eb2-86f4-76a6cbadec82 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 11 10:46:32.107310 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: warnings.warn(msg) Sep 11 10:46:32.107310 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: Sep 11 10:46:32.108546 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326d5364-4f23-4eb2-86f4-76a6cbadec82 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:32.112191 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 772/1555] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:46:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:46:32.113569 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f135761-dc33-4794-8f8c-69acb094fb98 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:32.123740 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f135761-dc33-4794-8f8c-69acb094fb98 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:32.128774 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a4c6d6-ba08-4996-80e7-097a77a71665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:32.129606 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a4c6d6-ba08-4996-80e7-097a77a71665 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:32.129889 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a4c6d6-ba08-4996-80e7-097a77a71665 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:32.130160 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a4c6d6-ba08-4996-80e7-097a77a71665 admin admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:32.130800 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a4c6d6-ba08-4996-80e7-097a77a71665 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:46:32.131100 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a4c6d6-ba08-4996-80e7-097a77a71665 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:32.134530 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 784/1556] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:46:32 2022] POST /identity/v3/users => generated 293 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:46:32.134530 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326d5364-4f23-4eb2-86f4-76a6cbadec82 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:32.134760 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a4c6d6-ba08-4996-80e7-097a77a71665 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:32.143024 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326d5364-4f23-4eb2-86f4-76a6cbadec82 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:32.151859 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 773/1557] 10.222.0.15 () {58 vars in 1153 bytes} [Sun Sep 11 10:46:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:46:32.151859 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ee75a-3a16-4640-b170-6470df9e0270 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:32.151859 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ee75a-3a16-4640-b170-6470df9e0270 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:32.151859 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ee75a-3a16-4640-b170-6470df9e0270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:32.152258 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8269ec98-1299-4945-9818-4bb017d72335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:32.152258 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8269ec98-1299-4945-9818-4bb017d72335 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:32.152258 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8269ec98-1299-4945-9818-4bb017d72335 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:32.152258 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8269ec98-1299-4945-9818-4bb017d72335 admin admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:32.152603 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8269ec98-1299-4945-9818-4bb017d72335 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:32.152603 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8269ec98-1299-4945-9818-4bb017d72335 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:32.166865 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-9b0ee75a-3a16-4640-b170-6470df9e0270 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:46:32.168743 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-9b0ee75a-3a16-4640-b170-6470df9e0270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:32.178112 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 774/1558] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:46:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:46:32.181892 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f78e4-5407-491a-8168-baa135ce3e02 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:32.181892 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f78e4-5407-491a-8168-baa135ce3e02 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:32.181892 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f78e4-5407-491a-8168-baa135ce3e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:32.186573 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 785/1559] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:46:32 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:46:32.187274 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8269ec98-1299-4945-9818-4bb017d72335 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:32.202327 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-5c3f78e4-5407-491a-8168-baa135ce3e02 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:46:32.203542 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-5c3f78e4-5407-491a-8168-baa135ce3e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:32.232620 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:46:32.250656 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8269ec98-1299-4945-9818-4bb017d72335 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:32.255872 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75595d37-0d25-47ad-8a56-ea4b776e073b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:32.256693 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75595d37-0d25-47ad-8a56-ea4b776e073b admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:32.256971 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75595d37-0d25-47ad-8a56-ea4b776e073b admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:32.257242 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75595d37-0d25-47ad-8a56-ea4b776e073b admin admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:32.257870 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75595d37-0d25-47ad-8a56-ea4b776e073b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:46:32.258178 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75595d37-0d25-47ad-8a56-ea4b776e073b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:32.259794 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75595d37-0d25-47ad-8a56-ea4b776e073b admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:32.264728 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 775/1560] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:46:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:46:32.266220 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56e4052-dfb4-4dee-8803-e70ec1cff5c6 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:32.266513 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56e4052-dfb4-4dee-8803-e70ec1cff5c6 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:32.266801 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56e4052-dfb4-4dee-8803-e70ec1cff5c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:32.280904 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 786/1561] 10.222.0.15 () {58 vars in 1153 bytes} [Sun Sep 11 10:46:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:46:32.280904 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75595d37-0d25-47ad-8a56-ea4b776e073b admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:32.286359 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75595d37-0d25-47ad-8a56-ea4b776e073b admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:32.289498 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-b56e4052-dfb4-4dee-8803-e70ec1cff5c6 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:46:32.290685 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-b56e4052-dfb4-4dee-8803-e70ec1cff5c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:32.291224 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3136d4f3-d8d8-4cff-b9d6-4b1aeb2c7578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:32.291862 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3136d4f3-d8d8-4cff-b9d6-4b1aeb2c7578 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:32.292142 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3136d4f3-d8d8-4cff-b9d6-4b1aeb2c7578 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:32.292414 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3136d4f3-d8d8-4cff-b9d6-4b1aeb2c7578 admin admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:32.293052 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3136d4f3-d8d8-4cff-b9d6-4b1aeb2c7578 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:46:32.293354 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3136d4f3-d8d8-4cff-b9d6-4b1aeb2c7578 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:32.294939 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3136d4f3-d8d8-4cff-b9d6-4b1aeb2c7578 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:32.298764 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 776/1562] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:46:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:46:32.299641 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56e4052-dfb4-4dee-8803-e70ec1cff5c6 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:32.309961 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 787/1563] 10.222.0.15 () {58 vars in 1153 bytes} [Sun Sep 11 10:46:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:46:32.309961 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3136d4f3-d8d8-4cff-b9d6-4b1aeb2c7578 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:32.310275 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56e4052-dfb4-4dee-8803-e70ec1cff5c6 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:32.315023 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a13b1fd-df59-48ed-bb57-196c9929c927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:32.315730 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a13b1fd-df59-48ed-bb57-196c9929c927 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:32.316136 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a13b1fd-df59-48ed-bb57-196c9929c927 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:32.316430 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a13b1fd-df59-48ed-bb57-196c9929c927 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:32.317084 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a13b1fd-df59-48ed-bb57-196c9929c927 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:32.318673 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a13b1fd-df59-48ed-bb57-196c9929c927 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:32.320943 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3136d4f3-d8d8-4cff-b9d6-4b1aeb2c7578 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:32.325554 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1734839c-bb27-430e-8bbe-b2d2ec27c341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:32.326142 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1734839c-bb27-430e-8bbe-b2d2ec27c341 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:32.326419 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1734839c-bb27-430e-8bbe-b2d2ec27c341 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:32.326727 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1734839c-bb27-430e-8bbe-b2d2ec27c341 admin admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:32.327335 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1734839c-bb27-430e-8bbe-b2d2ec27c341 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:32.331651 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 777/1564] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:46:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:46:32.331803 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1734839c-bb27-430e-8bbe-b2d2ec27c341 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:32.331899 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a13b1fd-df59-48ed-bb57-196c9929c927 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:32.339344 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a13b1fd-df59-48ed-bb57-196c9929c927 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:32.343862 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3250e62-010a-45af-a605-18259100a2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:32.344426 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3250e62-010a-45af-a605-18259100a2f1 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:32.344706 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3250e62-010a-45af-a605-18259100a2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:32.344975 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3250e62-010a-45af-a605-18259100a2f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:32.345643 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3250e62-010a-45af-a605-18259100a2f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:46:32.346122 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3250e62-010a-45af-a605-18259100a2f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:32.348843 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3250e62-010a-45af-a605-18259100a2f1 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:32.355922 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:46:32.357495 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 788/1565] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:46:32 2022] POST /identity/v3/projects => generated 389 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:46:32.358318 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1734839c-bb27-430e-8bbe-b2d2ec27c341 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:32.365059 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 778/1566] 10.222.0.15 () {58 vars in 1153 bytes} [Sun Sep 11 10:46:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:46:32.365892 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3250e62-010a-45af-a605-18259100a2f1 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:32.375989 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3250e62-010a-45af-a605-18259100a2f1 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:32.421632 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1734839c-bb27-430e-8bbe-b2d2ec27c341 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:32.426736 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec59dab-caa3-47a4-be4c-dca439126de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:32.427343 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec59dab-caa3-47a4-be4c-dca439126de7 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:32.427674 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec59dab-caa3-47a4-be4c-dca439126de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:32.428005 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec59dab-caa3-47a4-be4c-dca439126de7 admin admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:32.428626 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec59dab-caa3-47a4-be4c-dca439126de7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:32.430536 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec59dab-caa3-47a4-be4c-dca439126de7 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:32.433789 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac52235-9990-42a7-b890-a413b71d663f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:32.434395 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac52235-9990-42a7-b890-a413b71d663f admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:32.434673 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac52235-9990-42a7-b890-a413b71d663f admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:32.434942 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac52235-9990-42a7-b890-a413b71d663f admin admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:32.435610 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac52235-9990-42a7-b890-a413b71d663f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:46:32.435910 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac52235-9990-42a7-b890-a413b71d663f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:32.437473 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac52235-9990-42a7-b890-a413b71d663f admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:32.449789 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 779/1567] 10.222.0.15 () {58 vars in 1153 bytes} [Sun Sep 11 10:46:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:46:32.450568 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac52235-9990-42a7-b890-a413b71d663f admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:32.456022 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 789/1568] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:46:32 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:46:32.456022 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec59dab-caa3-47a4-be4c-dca439126de7 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:32.461760 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac52235-9990-42a7-b890-a413b71d663f admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:32.465560 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec59dab-caa3-47a4-be4c-dca439126de7 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:32.466396 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbf454e-d3b7-45ef-89bf-78b2d795aa79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:32.466986 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf454e-d3b7-45ef-89bf-78b2d795aa79 admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:32.467270 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf454e-d3b7-45ef-89bf-78b2d795aa79 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:32.467571 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf454e-d3b7-45ef-89bf-78b2d795aa79 admin admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:32.468199 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbf454e-d3b7-45ef-89bf-78b2d795aa79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:32.469202 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: WARNING py.warnings [None req-8bbf454e-d3b7-45ef-89bf-78b2d795aa79 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 11 10:46:32.469202 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: warnings.warn(msg) Sep 11 10:46:32.469202 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: Sep 11 10:46:32.470056 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f178da2-dd87-4850-9a3d-a46651d04f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:32.470379 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbf454e-d3b7-45ef-89bf-78b2d795aa79 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:32.470666 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f178da2-dd87-4850-9a3d-a46651d04f80 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:32.470946 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f178da2-dd87-4850-9a3d-a46651d04f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:32.471218 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f178da2-dd87-4850-9a3d-a46651d04f80 admin admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:32.471880 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f178da2-dd87-4850-9a3d-a46651d04f80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:46:32.472184 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f178da2-dd87-4850-9a3d-a46651d04f80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:32.474998 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f178da2-dd87-4850-9a3d-a46651d04f80 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:32.488185 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 790/1569] 10.222.0.15 () {58 vars in 1153 bytes} [Sun Sep 11 10:46:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:46:32.488185 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f178da2-dd87-4850-9a3d-a46651d04f80 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:32.498584 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f178da2-dd87-4850-9a3d-a46651d04f80 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:32.499576 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 780/1570] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:46:32 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:46:32.500417 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbf454e-d3b7-45ef-89bf-78b2d795aa79 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:32.558853 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f476571e-9c29-4a51-81c1-354c7819e41e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:32.559490 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f476571e-9c29-4a51-81c1-354c7819e41e admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:32.559775 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f476571e-9c29-4a51-81c1-354c7819e41e admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:32.560049 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f476571e-9c29-4a51-81c1-354c7819e41e admin admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:32.560675 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f476571e-9c29-4a51-81c1-354c7819e41e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:32.562468 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbf454e-d3b7-45ef-89bf-78b2d795aa79 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:32.562588 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f476571e-9c29-4a51-81c1-354c7819e41e admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:32.567466 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65330f1d-4d41-4611-ae8d-9ded4e950b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:32.568068 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65330f1d-4d41-4611-ae8d-9ded4e950b88 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:32.568495 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65330f1d-4d41-4611-ae8d-9ded4e950b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:32.568767 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65330f1d-4d41-4611-ae8d-9ded4e950b88 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:32.569401 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65330f1d-4d41-4611-ae8d-9ded4e950b88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:32.571020 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65330f1d-4d41-4611-ae8d-9ded4e950b88 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:32.582994 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 781/1571] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:46:32 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:46:32.583945 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65330f1d-4d41-4611-ae8d-9ded4e950b88 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:32.593403 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 791/1572] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:46:32 2022] POST /identity/v3/projects => generated 367 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:46:32.593403 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f476571e-9c29-4a51-81c1-354c7819e41e admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:32.596191 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:46:32.598102 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65330f1d-4d41-4611-ae8d-9ded4e950b88 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:32.652872 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f476571e-9c29-4a51-81c1-354c7819e41e admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:32.655943 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b0cbf-51e5-4314-9114-6f959472fac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:32.656559 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b0cbf-51e5-4314-9114-6f959472fac8 admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:32.656845 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b0cbf-51e5-4314-9114-6f959472fac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:32.657124 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b0cbf-51e5-4314-9114-6f959472fac8 admin admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:32.657555 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc5c4d3-d29c-4f7a-8a68-fe7cc2cf635c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:32.657748 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104b0cbf-51e5-4314-9114-6f959472fac8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:32.658158 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc5c4d3-d29c-4f7a-8a68-fe7cc2cf635c admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:32.658440 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc5c4d3-d29c-4f7a-8a68-fe7cc2cf635c admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:32.658713 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc5c4d3-d29c-4f7a-8a68-fe7cc2cf635c admin admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:32.659328 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc5c4d3-d29c-4f7a-8a68-fe7cc2cf635c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:32.659714 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104b0cbf-51e5-4314-9114-6f959472fac8 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:32.661518 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc5c4d3-d29c-4f7a-8a68-fe7cc2cf635c admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:32.670874 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:46:32.686571 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 782/1573] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:46:32 2022] POST /identity/v3/users => generated 295 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:46:32.687708 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b0cbf-51e5-4314-9114-6f959472fac8 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:32.690151 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 792/1574] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:46:32 2022] POST /identity/v3/projects => generated 373 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:46:32.693249 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc5c4d3-d29c-4f7a-8a68-fe7cc2cf635c admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:32.697689 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b0cbf-51e5-4314-9114-6f959472fac8 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:32.755291 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b3554f-f2e2-44c0-9d80-275b5583ef48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:32.755936 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b3554f-f2e2-44c0-9d80-275b5583ef48 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:32.756217 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b3554f-f2e2-44c0-9d80-275b5583ef48 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:32.756492 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b3554f-f2e2-44c0-9d80-275b5583ef48 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:32.756894 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc5c4d3-d29c-4f7a-8a68-fe7cc2cf635c admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:32.757129 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b3554f-f2e2-44c0-9d80-275b5583ef48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:32.758730 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b3554f-f2e2-44c0-9d80-275b5583ef48 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:32.761641 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70531fe6-8092-4987-81ac-847f096144d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:32.762250 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70531fe6-8092-4987-81ac-847f096144d4 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:32.762533 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70531fe6-8092-4987-81ac-847f096144d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:32.762810 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70531fe6-8092-4987-81ac-847f096144d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:32.763459 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70531fe6-8092-4987-81ac-847f096144d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:32.765371 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70531fe6-8092-4987-81ac-847f096144d4 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:32.768399 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 783/1575] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:46:32 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:46:32.769735 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b3554f-f2e2-44c0-9d80-275b5583ef48 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:32.779461 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b3554f-f2e2-44c0-9d80-275b5583ef48 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:32.784458 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18527ce9-7925-4dae-88c1-806692d54034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:32.785600 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18527ce9-7925-4dae-88c1-806692d54034 admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:32.785881 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18527ce9-7925-4dae-88c1-806692d54034 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:32.786163 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18527ce9-7925-4dae-88c1-806692d54034 admin admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:32.786821 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18527ce9-7925-4dae-88c1-806692d54034 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:32.788719 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18527ce9-7925-4dae-88c1-806692d54034 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:32.793769 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 793/1576] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:46:32 2022] POST /identity/v3/projects => generated 377 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:46:32.794563 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70531fe6-8092-4987-81ac-847f096144d4 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:32.820345 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 784/1577] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:46:32 2022] POST /identity/v3/projects => generated 391 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:46:32.820345 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18527ce9-7925-4dae-88c1-806692d54034 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:32.857377 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70531fe6-8092-4987-81ac-847f096144d4 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:32.875239 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:46:32.881283 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18527ce9-7925-4dae-88c1-806692d54034 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:32.885988 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81bf674-be22-40d8-8f1b-04536b2946bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:32.886624 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81bf674-be22-40d8-8f1b-04536b2946bf admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:32.886928 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81bf674-be22-40d8-8f1b-04536b2946bf admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:32.887226 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81bf674-be22-40d8-8f1b-04536b2946bf admin admin] PATH_INFO: `/v3/projects/cfc31fd786a54975ae40a9f4c1c2c8e4/users/2540f47512f74979948ba4ffa3f0415f/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:32.902404 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81bf674-be22-40d8-8f1b-04536b2946bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfc31fd786a54975ae40a9f4c1c2c8e4, user_id=2540f47512f74979948ba4ffa3f0415f, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:32.903525 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: WARNING py.warnings [None req-c81bf674-be22-40d8-8f1b-04536b2946bf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 11 10:46:32.903525 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: warnings.warn(msg) Sep 11 10:46:32.903525 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: Sep 11 10:46:32.905456 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81bf674-be22-40d8-8f1b-04536b2946bf admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:32.918219 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1d733e-5c8c-436d-8f98-9f44c97dd8a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:32.918840 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d733e-5c8c-436d-8f98-9f44c97dd8a7 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:32.919118 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d733e-5c8c-436d-8f98-9f44c97dd8a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:32.919390 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d733e-5c8c-436d-8f98-9f44c97dd8a7 admin admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:32.920042 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1d733e-5c8c-436d-8f98-9f44c97dd8a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:32.920275 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 785/1578] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:46:32 2022] PUT /identity/v3/projects/cfc31fd786a54975ae40a9f4c1c2c8e4/users/2540f47512f74979948ba4ffa3f0415f/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:46:32.931990 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81bf674-be22-40d8-8f1b-04536b2946bf admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:32.932123 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1d733e-5c8c-436d-8f98-9f44c97dd8a7 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:32.945690 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 794/1579] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:46:32 2022] POST /identity/v3/users => generated 289 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:46:32.946919 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1d733e-5c8c-436d-8f98-9f44c97dd8a7 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:32.956848 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1d733e-5c8c-436d-8f98-9f44c97dd8a7 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:32.981879 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:46:32.994565 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81bf674-be22-40d8-8f1b-04536b2946bf admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:32.999576 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2733098e-a86b-4f45-999c-f6150d9d8a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:33.000186 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2733098e-a86b-4f45-999c-f6150d9d8a18 admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:33.000462 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2733098e-a86b-4f45-999c-f6150d9d8a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:33.000728 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2733098e-a86b-4f45-999c-f6150d9d8a18 admin admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:33.001345 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2733098e-a86b-4f45-999c-f6150d9d8a18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:33.003253 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2733098e-a86b-4f45-999c-f6150d9d8a18 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:33.014464 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe5076c-22e0-4a10-9569-986eacf60dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:33.015079 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe5076c-22e0-4a10-9569-986eacf60dca admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:33.015362 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe5076c-22e0-4a10-9569-986eacf60dca admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:33.015675 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe5076c-22e0-4a10-9569-986eacf60dca admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:33.016317 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe5076c-22e0-4a10-9569-986eacf60dca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:33.017952 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe5076c-22e0-4a10-9569-986eacf60dca admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:33.025105 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 786/1580] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:46:32 2022] POST /identity/v3/users => generated 284 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:46:33.026142 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2733098e-a86b-4f45-999c-f6150d9d8a18 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:33.028390 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 795/1581] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:46:32 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:46:33.029211 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe5076c-22e0-4a10-9569-986eacf60dca admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:33.036989 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2733098e-a86b-4f45-999c-f6150d9d8a18 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:33.040482 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe5076c-22e0-4a10-9569-986eacf60dca admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:33.041536 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce64a378-e286-407a-80ee-605619996499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:33.042128 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce64a378-e286-407a-80ee-605619996499 admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:33.042403 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce64a378-e286-407a-80ee-605619996499 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:33.042671 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce64a378-e286-407a-80ee-605619996499 admin admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:33.043279 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce64a378-e286-407a-80ee-605619996499 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:33.045218 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce64a378-e286-407a-80ee-605619996499 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:33.045345 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e37c88-d897-4593-bd88-3ae6ccd29132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:33.045926 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e37c88-d897-4593-bd88-3ae6ccd29132 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:33.046213 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e37c88-d897-4593-bd88-3ae6ccd29132 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:33.046484 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e37c88-d897-4593-bd88-3ae6ccd29132 admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:33.047141 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e37c88-d897-4593-bd88-3ae6ccd29132 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:33.048745 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e37c88-d897-4593-bd88-3ae6ccd29132 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:33.059173 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 796/1582] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:46:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:46:33.063083 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e37c88-d897-4593-bd88-3ae6ccd29132 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:33.067612 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 787/1583] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:46:33 2022] POST /identity/v3/users => generated 287 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:46:33.068398 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce64a378-e286-407a-80ee-605619996499 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:33.073624 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e37c88-d897-4593-bd88-3ae6ccd29132 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:33.078258 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe9f2a9-7fc9-41bb-a435-71a933f6eae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:33.078507 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce64a378-e286-407a-80ee-605619996499 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:33.078836 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe9f2a9-7fc9-41bb-a435-71a933f6eae2 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:33.079143 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe9f2a9-7fc9-41bb-a435-71a933f6eae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:33.079377 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe9f2a9-7fc9-41bb-a435-71a933f6eae2 admin admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:33.080023 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe9f2a9-7fc9-41bb-a435-71a933f6eae2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:33.082575 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe9f2a9-7fc9-41bb-a435-71a933f6eae2 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:33.084238 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a549ec78-f318-4e0e-8cb3-a4bb5d375c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:33.084827 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a549ec78-f318-4e0e-8cb3-a4bb5d375c1e admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:33.085141 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a549ec78-f318-4e0e-8cb3-a4bb5d375c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:33.085417 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a549ec78-f318-4e0e-8cb3-a4bb5d375c1e admin admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:33.086041 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a549ec78-f318-4e0e-8cb3-a4bb5d375c1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:33.087999 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a549ec78-f318-4e0e-8cb3-a4bb5d375c1e admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:33.107451 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 797/1584] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:46:33 2022] POST /identity/v3/users => generated 289 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:46:33.107451 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe55874-6b95-4954-b3c7-0f19595e59e5 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:33.107785 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe55874-6b95-4954-b3c7-0f19595e59e5 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:33.107996 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe55874-6b95-4954-b3c7-0f19595e59e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:33.116509 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 788/1585] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:46:33 2022] POST /identity/v3/users => generated 296 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:46:33.116509 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a549ec78-f318-4e0e-8cb3-a4bb5d375c1e admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:33.123984 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a549ec78-f318-4e0e-8cb3-a4bb5d375c1e admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:33.129082 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f6be02-8e63-4e69-a178-b77090316ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:33.129676 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f6be02-8e63-4e69-a178-b77090316ff3 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:33.129958 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f6be02-8e63-4e69-a178-b77090316ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:33.130231 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f6be02-8e63-4e69-a178-b77090316ff3 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:33.130870 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f6be02-8e63-4e69-a178-b77090316ff3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:33.132496 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f6be02-8e63-4e69-a178-b77090316ff3 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:33.139691 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-ebe55874-6b95-4954-b3c7-0f19595e59e5 None None] MFA Rules not processed for user `2540f47512f74979948ba4ffa3f0415f`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:46:33.140908 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-ebe55874-6b95-4954-b3c7-0f19595e59e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:33.142268 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 789/1586] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:46:33 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:46:33.143053 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f6be02-8e63-4e69-a178-b77090316ff3 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:33.152976 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f6be02-8e63-4e69-a178-b77090316ff3 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:33.157661 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52942e9f-8b93-415b-b1a1-a24136de589e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:33.158181 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52942e9f-8b93-415b-b1a1-a24136de589e admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:33.158464 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52942e9f-8b93-415b-b1a1-a24136de589e admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:33.158740 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52942e9f-8b93-415b-b1a1-a24136de589e admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:33.159366 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52942e9f-8b93-415b-b1a1-a24136de589e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:33.160997 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52942e9f-8b93-415b-b1a1-a24136de589e admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:33.170991 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 790/1587] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:46:33 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:46:33.171793 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52942e9f-8b93-415b-b1a1-a24136de589e admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:33.181196 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52942e9f-8b93-415b-b1a1-a24136de589e admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:33.185843 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08c90ee-698a-4ce3-80c7-bc60316addb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:33.186421 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c90ee-698a-4ce3-80c7-bc60316addb7 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:33.186707 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c90ee-698a-4ce3-80c7-bc60316addb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:33.186988 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c90ee-698a-4ce3-80c7-bc60316addb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:33.187641 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08c90ee-698a-4ce3-80c7-bc60316addb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:33.189211 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08c90ee-698a-4ce3-80c7-bc60316addb7 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:33.200016 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 791/1588] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:46:33 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:46:33.200781 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08c90ee-698a-4ce3-80c7-bc60316addb7 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:33.209232 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 798/1589] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:46:33 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:46:33.210069 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe55874-6b95-4954-b3c7-0f19595e59e5 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:33.210662 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08c90ee-698a-4ce3-80c7-bc60316addb7 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:33.217039 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291d570e-ae6e-4f11-9138-dccdebdf51e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:33.218186 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291d570e-ae6e-4f11-9138-dccdebdf51e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:33.219158 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291d570e-ae6e-4f11-9138-dccdebdf51e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:33.219759 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291d570e-ae6e-4f11-9138-dccdebdf51e7 admin admin] PATH_INFO: `/v3/projects/7ffde984c6d34c33a44e498ccaa3b668/users/7568a199ca0742f09b1e0e02c40bd14a/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:33.223217 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe55874-6b95-4954-b3c7-0f19595e59e5 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:33.229199 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156828a6-31c1-4b18-aaf8-ca082497e732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:33.229815 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156828a6-31c1-4b18-aaf8-ca082497e732 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:33.230099 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156828a6-31c1-4b18-aaf8-ca082497e732 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:33.230637 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156828a6-31c1-4b18-aaf8-ca082497e732 admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:33.231068 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156828a6-31c1-4b18-aaf8-ca082497e732 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:33.233844 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156828a6-31c1-4b18-aaf8-ca082497e732 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:33.236068 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:46:33.237036 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291d570e-ae6e-4f11-9138-dccdebdf51e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ffde984c6d34c33a44e498ccaa3b668, user_id=7568a199ca0742f09b1e0e02c40bd14a, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:33.242671 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291d570e-ae6e-4f11-9138-dccdebdf51e7 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:33.247303 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 799/1590] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:46:33 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:46:33.251347 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156828a6-31c1-4b18-aaf8-ca082497e732 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:33.262568 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156828a6-31c1-4b18-aaf8-ca082497e732 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:33.262778 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 792/1591] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:46:33 2022] PUT /identity/v3/projects/7ffde984c6d34c33a44e498ccaa3b668/users/7568a199ca0742f09b1e0e02c40bd14a/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:46:33.263607 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291d570e-ae6e-4f11-9138-dccdebdf51e7 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:33.328707 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9a6ba9-6c67-4a87-affd-2b8956deae35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:33.329321 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a6ba9-6c67-4a87-affd-2b8956deae35 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:33.329604 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a6ba9-6c67-4a87-affd-2b8956deae35 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:33.329870 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a6ba9-6c67-4a87-affd-2b8956deae35 admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:33.330520 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9a6ba9-6c67-4a87-affd-2b8956deae35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:33.332562 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9a6ba9-6c67-4a87-affd-2b8956deae35 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:33.333023 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291d570e-ae6e-4f11-9138-dccdebdf51e7 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:33.337706 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96d7b95-c827-4abc-9e96-f7c2922b985b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:33.338315 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d7b95-c827-4abc-9e96-f7c2922b985b admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:33.338595 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d7b95-c827-4abc-9e96-f7c2922b985b admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:33.338868 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d7b95-c827-4abc-9e96-f7c2922b985b admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:33.339535 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96d7b95-c827-4abc-9e96-f7c2922b985b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:33.341118 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96d7b95-c827-4abc-9e96-f7c2922b985b admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:33.342100 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 800/1592] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:46:33 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:46:33.342878 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9a6ba9-6c67-4a87-affd-2b8956deae35 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:33.351808 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 793/1593] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:46:33 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:46:33.352625 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96d7b95-c827-4abc-9e96-f7c2922b985b admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:33.353014 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9a6ba9-6c67-4a87-affd-2b8956deae35 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:33.357886 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8fd840-6bb2-472e-aaaf-af529fbcc0b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:33.358472 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8fd840-6bb2-472e-aaaf-af529fbcc0b3 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:33.358749 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8fd840-6bb2-472e-aaaf-af529fbcc0b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:33.359018 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8fd840-6bb2-472e-aaaf-af529fbcc0b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:33.359667 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8fd840-6bb2-472e-aaaf-af529fbcc0b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:33.361235 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8fd840-6bb2-472e-aaaf-af529fbcc0b3 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:33.362498 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96d7b95-c827-4abc-9e96-f7c2922b985b admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:33.367685 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1f0145-e740-4efa-80c0-133ecb4c5272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:33.368320 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f0145-e740-4efa-80c0-133ecb4c5272 admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:33.368603 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f0145-e740-4efa-80c0-133ecb4c5272 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:33.368885 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f0145-e740-4efa-80c0-133ecb4c5272 admin admin] PATH_INFO: `/v3/projects/d0f44d45f87c4b28a1672c48fd9613ca/users/2238a62cbf1846999b31ea9434b67743/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:33.375870 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 801/1594] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:46:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:46:33.375870 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8fd840-6bb2-472e-aaaf-af529fbcc0b3 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:33.382927 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8fd840-6bb2-472e-aaaf-af529fbcc0b3 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:33.388249 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb6b7ea-1bfb-4f87-bffe-f4112ba8ac86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:33.388847 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb6b7ea-1bfb-4f87-bffe-f4112ba8ac86 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:33.389048 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1f0145-e740-4efa-80c0-133ecb4c5272 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0f44d45f87c4b28a1672c48fd9613ca, user_id=2238a62cbf1846999b31ea9434b67743, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:33.389221 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb6b7ea-1bfb-4f87-bffe-f4112ba8ac86 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:33.389381 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb6b7ea-1bfb-4f87-bffe-f4112ba8ac86 admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:33.390059 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb6b7ea-1bfb-4f87-bffe-f4112ba8ac86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:33.391553 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1f0145-e740-4efa-80c0-133ecb4c5272 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:33.391761 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb6b7ea-1bfb-4f87-bffe-f4112ba8ac86 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:33.406166 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 802/1595] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:46:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:46:33.406166 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0df18ec-2e1c-4501-9732-5d0cae2a7aec None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:33.406166 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0df18ec-2e1c-4501-9732-5d0cae2a7aec None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:33.406166 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0df18ec-2e1c-4501-9732-5d0cae2a7aec None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:33.407158 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 803/1596] 10.222.0.15 () {54 vars in 962 bytes} [Sun Sep 11 10:46:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:46:33.408032 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0df18ec-2e1c-4501-9732-5d0cae2a7aec None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:33.411299 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 794/1597] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:46:33 2022] PUT /identity/v3/projects/d0f44d45f87c4b28a1672c48fd9613ca/users/2238a62cbf1846999b31ea9434b67743/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:46:33.412743 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47515860-f411-43dc-9a5d-e57172863312 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:33.413034 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47515860-f411-43dc-9a5d-e57172863312 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:33.413314 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47515860-f411-43dc-9a5d-e57172863312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:33.440423 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-47515860-f411-43dc-9a5d-e57172863312 None None] MFA Rules not processed for user `7568a199ca0742f09b1e0e02c40bd14a`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:46:33.441844 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-47515860-f411-43dc-9a5d-e57172863312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:33.450782 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=89,keystone:INSERT=11 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:46:33.478960 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0df18ec-2e1c-4501-9732-5d0cae2a7aec None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:33.483837 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222670cb-d5c7-4da7-8433-747e671fa9e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:33.484447 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222670cb-d5c7-4da7-8433-747e671fa9e5 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:33.484729 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222670cb-d5c7-4da7-8433-747e671fa9e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:33.485003 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222670cb-d5c7-4da7-8433-747e671fa9e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:33.485643 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222670cb-d5c7-4da7-8433-747e671fa9e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:33.487242 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222670cb-d5c7-4da7-8433-747e671fa9e5 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:33.497936 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 804/1598] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:46:33 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:46:33.498894 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222670cb-d5c7-4da7-8433-747e671fa9e5 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:33.508971 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222670cb-d5c7-4da7-8433-747e671fa9e5 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:33.513588 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03fe01b-36b2-4a39-9afe-46d06f40b30d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:33.514033 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 795/1599] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:46:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:46:33.514257 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03fe01b-36b2-4a39-9afe-46d06f40b30d admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:33.514666 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03fe01b-36b2-4a39-9afe-46d06f40b30d admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:33.514775 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47515860-f411-43dc-9a5d-e57172863312 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:33.514955 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03fe01b-36b2-4a39-9afe-46d06f40b30d admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:33.515627 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03fe01b-36b2-4a39-9afe-46d06f40b30d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:33.517330 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03fe01b-36b2-4a39-9afe-46d06f40b30d admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:33.525564 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47515860-f411-43dc-9a5d-e57172863312 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:33.528706 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 805/1600] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:46:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:46:33.529448 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03fe01b-36b2-4a39-9afe-46d06f40b30d admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:33.533577 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c824a245-0f2c-4b15-a7c3-8377690f49c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:33.534415 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c824a245-0f2c-4b15-a7c3-8377690f49c4 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:33.534724 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c824a245-0f2c-4b15-a7c3-8377690f49c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:33.535032 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c824a245-0f2c-4b15-a7c3-8377690f49c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:33.535790 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c824a245-0f2c-4b15-a7c3-8377690f49c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:33.537486 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c824a245-0f2c-4b15-a7c3-8377690f49c4 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:33.539156 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03fe01b-36b2-4a39-9afe-46d06f40b30d admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:33.543674 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feee8e48-3ea4-4b84-8227-778d8a015551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:33.544289 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee8e48-3ea4-4b84-8227-778d8a015551 admin admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:33.544560 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee8e48-3ea4-4b84-8227-778d8a015551 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:33.544830 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee8e48-3ea4-4b84-8227-778d8a015551 admin admin] PATH_INFO: `/v3/projects/204ecef1ba96491cbfb3e8176e6c22ec/users/8d0af2b5f354439d92bfa317a0baefd5/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:33.547707 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 796/1601] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:46:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:46:33.548481 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c824a245-0f2c-4b15-a7c3-8377690f49c4 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:33.558931 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c824a245-0f2c-4b15-a7c3-8377690f49c4 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:33.562127 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feee8e48-3ea4-4b84-8227-778d8a015551 admin admin] RBAC: Authorizing `identity:create_grant(project_id=204ecef1ba96491cbfb3e8176e6c22ec, user_id=8d0af2b5f354439d92bfa317a0baefd5, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:33.563517 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22166b46-8d64-44aa-9e62-0f38903eaa6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:33.564190 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22166b46-8d64-44aa-9e62-0f38903eaa6b admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:33.564468 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22166b46-8d64-44aa-9e62-0f38903eaa6b admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:33.564739 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22166b46-8d64-44aa-9e62-0f38903eaa6b admin admin] PATH_INFO: `/v3/projects/d8af004e81e544519c5d040babb51fe1/users/fe55415017ae433cb76b49b164a13ef6/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:33.564837 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feee8e48-3ea4-4b84-8227-778d8a015551 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:33.578425 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 806/1602] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:46:33 2022] PUT /identity/v3/projects/204ecef1ba96491cbfb3e8176e6c22ec/users/8d0af2b5f354439d92bfa317a0baefd5/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:46:33.579652 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b26bd-ece8-4800-9ea3-d821f26b09ca None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:33.579931 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b26bd-ece8-4800-9ea3-d821f26b09ca None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:33.580198 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b26bd-ece8-4800-9ea3-d821f26b09ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:33.584701 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22166b46-8d64-44aa-9e62-0f38903eaa6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8af004e81e544519c5d040babb51fe1, user_id=fe55415017ae433cb76b49b164a13ef6, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:33.587399 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22166b46-8d64-44aa-9e62-0f38903eaa6b admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:33.601454 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 797/1603] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:46:33 2022] PUT /identity/v3/projects/d8af004e81e544519c5d040babb51fe1/users/fe55415017ae433cb76b49b164a13ef6/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:46:33.603575 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3992f482-1973-4598-aa15-1c0012ef5e8c None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:33.603866 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3992f482-1973-4598-aa15-1c0012ef5e8c None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:33.604140 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3992f482-1973-4598-aa15-1c0012ef5e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:33.623812 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-3992f482-1973-4598-aa15-1c0012ef5e8c None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:46:33.625026 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-3992f482-1973-4598-aa15-1c0012ef5e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:33.627204 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-5d0b26bd-ece8-4800-9ea3-d821f26b09ca None None] MFA Rules not processed for user `2bd5356829b6454993bab9bad2f79fed`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:46:33.628515 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-5d0b26bd-ece8-4800-9ea3-d821f26b09ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:33.641605 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:46:33.684507 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 798/1604] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:46:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:46:33.685181 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3992f482-1973-4598-aa15-1c0012ef5e8c None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:33.696199 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3992f482-1973-4598-aa15-1c0012ef5e8c None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:33.701368 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49830997-4fc6-4f6d-9619-66fe91e942c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:33.702036 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49830997-4fc6-4f6d-9619-66fe91e942c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:33.702321 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49830997-4fc6-4f6d-9619-66fe91e942c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:33.702599 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49830997-4fc6-4f6d-9619-66fe91e942c7 admin admin] PATH_INFO: `/v3/projects/22d7f01aefde4648ac6332e67d622214/users/1eb7a55cacf3400cbbdead369f9f65e7/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:33.702879 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 807/1605] 10.222.0.15 () {58 vars in 1059 bytes} [Sun Sep 11 10:46:33 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:46:33.704032 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b26bd-ece8-4800-9ea3-d821f26b09ca None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:33.715023 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b26bd-ece8-4800-9ea3-d821f26b09ca None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:33.718520 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49830997-4fc6-4f6d-9619-66fe91e942c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22d7f01aefde4648ac6332e67d622214, user_id=1eb7a55cacf3400cbbdead369f9f65e7, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:33.719607 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e0ada5-c6c0-49bb-bfeb-9d3dfc97bef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:33.720490 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e0ada5-c6c0-49bb-bfeb-9d3dfc97bef4 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:33.720776 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e0ada5-c6c0-49bb-bfeb-9d3dfc97bef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:33.721054 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e0ada5-c6c0-49bb-bfeb-9d3dfc97bef4 admin admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:33.721233 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49830997-4fc6-4f6d-9619-66fe91e942c7 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:33.721682 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e0ada5-c6c0-49bb-bfeb-9d3dfc97bef4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:33.724133 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e0ada5-c6c0-49bb-bfeb-9d3dfc97bef4 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:33.736044 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 799/1606] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:46:33 2022] PUT /identity/v3/projects/22d7f01aefde4648ac6332e67d622214/users/1eb7a55cacf3400cbbdead369f9f65e7/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:46:33.736864 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49830997-4fc6-4f6d-9619-66fe91e942c7 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:33.769708 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 808/1607] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:46:33 2022] POST /identity/v3/projects => generated 371 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:46:33.769708 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e0ada5-c6c0-49bb-bfeb-9d3dfc97bef4 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:33.793556 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:46:33.806374 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49830997-4fc6-4f6d-9619-66fe91e942c7 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:33.829288 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e0ada5-c6c0-49bb-bfeb-9d3dfc97bef4 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:33.834380 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3786b416-2874-49c9-b6b0-2017d3e514b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:33.835055 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3786b416-2874-49c9-b6b0-2017d3e514b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:33.835327 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3786b416-2874-49c9-b6b0-2017d3e514b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:33.835633 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3786b416-2874-49c9-b6b0-2017d3e514b1 admin admin] PATH_INFO: `/v3/projects/528ee82fc48247eea01a17898488ea3e/users/f6eaba3a278e4bc99aca0dedd649fa9f/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:33.853400 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3786b416-2874-49c9-b6b0-2017d3e514b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=528ee82fc48247eea01a17898488ea3e, user_id=f6eaba3a278e4bc99aca0dedd649fa9f, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:33.856152 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3786b416-2874-49c9-b6b0-2017d3e514b1 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:33.867554 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530879a3-d2c8-4822-8471-94de7d60de25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:33.868230 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530879a3-d2c8-4822-8471-94de7d60de25 admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:33.868510 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530879a3-d2c8-4822-8471-94de7d60de25 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:33.868783 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530879a3-d2c8-4822-8471-94de7d60de25 admin admin] PATH_INFO: `/v3/projects/74e895a982b34e958af26f5287567949/users/ea9f58a33d1449099bd39f8d09f9f0cd/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:33.869649 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 809/1608] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:46:33 2022] PUT /identity/v3/projects/528ee82fc48247eea01a17898488ea3e/users/f6eaba3a278e4bc99aca0dedd649fa9f/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:46:33.870871 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f428e2-6954-4cfb-a127-d60836bca4fd None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:33.871159 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f428e2-6954-4cfb-a127-d60836bca4fd None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:33.871454 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f428e2-6954-4cfb-a127-d60836bca4fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:33.888390 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530879a3-d2c8-4822-8471-94de7d60de25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74e895a982b34e958af26f5287567949, user_id=ea9f58a33d1449099bd39f8d09f9f0cd, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:33.891062 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530879a3-d2c8-4822-8471-94de7d60de25 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:33.891394 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-59f428e2-6954-4cfb-a127-d60836bca4fd None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:46:33.892639 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-59f428e2-6954-4cfb-a127-d60836bca4fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:33.904445 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 800/1609] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:46:33 2022] PUT /identity/v3/projects/74e895a982b34e958af26f5287567949/users/ea9f58a33d1449099bd39f8d09f9f0cd/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:46:33.905877 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7bfe79-38ee-415a-a7b7-281297aa29fd None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:33.906282 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7bfe79-38ee-415a-a7b7-281297aa29fd None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:33.906662 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7bfe79-38ee-415a-a7b7-281297aa29fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:33.934787 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-ba7bfe79-38ee-415a-a7b7-281297aa29fd None None] MFA Rules not processed for user `fe55415017ae433cb76b49b164a13ef6`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:46:33.935944 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-ba7bfe79-38ee-415a-a7b7-281297aa29fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:33.954539 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 810/1610] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:46:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:46:33.958776 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f428e2-6954-4cfb-a127-d60836bca4fd None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:33.971701 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:46:34.002820 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 801/1611] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:46:33 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:46:34.004305 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7bfe79-38ee-415a-a7b7-281297aa29fd None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:34.020872 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f428e2-6954-4cfb-a127-d60836bca4fd None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:34.069574 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920d914c-5279-4daa-8254-7919013c28f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:34.070188 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920d914c-5279-4daa-8254-7919013c28f1 service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:34.070468 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920d914c-5279-4daa-8254-7919013c28f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:34.070741 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920d914c-5279-4daa-8254-7919013c28f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:34.076894 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7bfe79-38ee-415a-a7b7-281297aa29fd None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:34.082744 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760961a6-b356-45f6-8ea8-7eed933c0eaa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bd5356829b6454993bab9bad2f79fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:34.083346 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760961a6-b356-45f6-8ea8-7eed933c0eaa service nova] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:34.083654 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760961a6-b356-45f6-8ea8-7eed933c0eaa service nova] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:34.083927 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760961a6-b356-45f6-8ea8-7eed933c0eaa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:34.096257 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:46:34.132713 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920d914c-5279-4daa-8254-7919013c28f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:34.133934 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920d914c-5279-4daa-8254-7919013c28f1 service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:34.143630 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 811/1612] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:46:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:46:34.145437 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5472ff8b-152f-4f8b-a872-f4348d5a619c None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:34.145876 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5472ff8b-152f-4f8b-a872-f4348d5a619c None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:34.146156 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5472ff8b-152f-4f8b-a872-f4348d5a619c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:34.149664 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760961a6-b356-45f6-8ea8-7eed933c0eaa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:34.151395 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760961a6-b356-45f6-8ea8-7eed933c0eaa service nova] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:34.161044 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 802/1613] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:46:34 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:46:34.161854 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760961a6-b356-45f6-8ea8-7eed933c0eaa service nova] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:34.172614 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760961a6-b356-45f6-8ea8-7eed933c0eaa service nova] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:34.177237 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-5472ff8b-152f-4f8b-a872-f4348d5a619c None None] MFA Rules not processed for user `1eb7a55cacf3400cbbdead369f9f65e7`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:46:34.178411 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be84f298-a071-46fe-ad5c-5b7b4f604da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:34.178546 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-5472ff8b-152f-4f8b-a872-f4348d5a619c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:34.179082 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be84f298-a071-46fe-ad5c-5b7b4f604da6 admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:34.179342 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be84f298-a071-46fe-ad5c-5b7b4f604da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:34.179642 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be84f298-a071-46fe-ad5c-5b7b4f604da6 admin admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:34.180263 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be84f298-a071-46fe-ad5c-5b7b4f604da6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:34.182175 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be84f298-a071-46fe-ad5c-5b7b4f604da6 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:34.207503 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 803/1614] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:46:34 2022] POST /identity/v3/users => generated 292 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:46:34.208836 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aed962-7bdc-4a66-a1d4-994220aaca24 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:34.209576 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aed962-7bdc-4a66-a1d4-994220aaca24 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:34.209941 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aed962-7bdc-4a66-a1d4-994220aaca24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:34.232679 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-17aed962-7bdc-4a66-a1d4-994220aaca24 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:46:34.234226 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-17aed962-7bdc-4a66-a1d4-994220aaca24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:34.244292 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 804/1615] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:46:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:46:34.245595 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5eada4-f2c1-4b04-8678-adc27bae95e7 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:34.245880 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5eada4-f2c1-4b04-8678-adc27bae95e7 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:34.246145 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5eada4-f2c1-4b04-8678-adc27bae95e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:34.269151 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-9a5eada4-f2c1-4b04-8678-adc27bae95e7 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:46:34.270809 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-9a5eada4-f2c1-4b04-8678-adc27bae95e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:34.282560 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 805/1616] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:46:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:46:34.282560 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5eada4-f2c1-4b04-8678-adc27bae95e7 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:34.290760 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5eada4-f2c1-4b04-8678-adc27bae95e7 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:34.294568 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fcdfd2-0bc1-4532-8082-433cf65eccac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:34.295179 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fcdfd2-0bc1-4532-8082-433cf65eccac service neutron] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:34.295545 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fcdfd2-0bc1-4532-8082-433cf65eccac service neutron] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:34.295825 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fcdfd2-0bc1-4532-8082-433cf65eccac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:34.300175 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 812/1617] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:46:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:46:34.302765 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5472ff8b-152f-4f8b-a872-f4348d5a619c None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:34.310646 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fcdfd2-0bc1-4532-8082-433cf65eccac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:34.312685 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fcdfd2-0bc1-4532-8082-433cf65eccac service neutron] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:34.320426 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:46:34.327831 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 806/1618] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:46:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:46:34.328623 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fcdfd2-0bc1-4532-8082-433cf65eccac service neutron] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:34.341596 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fcdfd2-0bc1-4532-8082-433cf65eccac service neutron] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:34.347466 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcec4675-27df-4e85-bac2-99fe03aeade1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:34.348102 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec4675-27df-4e85-bac2-99fe03aeade1 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:34.348785 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec4675-27df-4e85-bac2-99fe03aeade1 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:34.349072 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec4675-27df-4e85-bac2-99fe03aeade1 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:34.349720 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcec4675-27df-4e85-bac2-99fe03aeade1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:34.351680 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcec4675-27df-4e85-bac2-99fe03aeade1 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:34.372119 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 807/1619] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:46:34 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:46:34.373223 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcec4675-27df-4e85-bac2-99fe03aeade1 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:34.380769 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-bcec4675-27df-4e85-bac2-99fe03aeade1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:34.396195 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5472ff8b-152f-4f8b-a872-f4348d5a619c None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:34.400328 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c182c281-17f3-4b80-b7d7-17282ec7f415 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:34.401310 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c182c281-17f3-4b80-b7d7-17282ec7f415 service cinder] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:34.401739 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c182c281-17f3-4b80-b7d7-17282ec7f415 service cinder] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:34.402189 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c182c281-17f3-4b80-b7d7-17282ec7f415 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:34.402669 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcec4675-27df-4e85-bac2-99fe03aeade1 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:34.407191 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1bdaad-31e1-4067-9a11-03ab9a513ecd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:34.408108 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1bdaad-31e1-4067-9a11-03ab9a513ecd service neutron] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:34.408540 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1bdaad-31e1-4067-9a11-03ab9a513ecd service neutron] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:34.408930 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1bdaad-31e1-4067-9a11-03ab9a513ecd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:34.422390 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c182c281-17f3-4b80-b7d7-17282ec7f415 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:34.423745 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1bdaad-31e1-4067-9a11-03ab9a513ecd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:34.424978 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c182c281-17f3-4b80-b7d7-17282ec7f415 service cinder] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:34.426406 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1bdaad-31e1-4067-9a11-03ab9a513ecd service neutron] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:34.447944 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 813/1620] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:46:34 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:46:34.459800 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 808/1621] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:46:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:46:34.460335 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c182c281-17f3-4b80-b7d7-17282ec7f415 service cinder] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:34.462496 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca307e-01fe-4a5b-8014-0c5f3a5de972 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:34.463130 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca307e-01fe-4a5b-8014-0c5f3a5de972 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:34.463613 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca307e-01fe-4a5b-8014-0c5f3a5de972 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:34.465278 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 809/1622] 10.222.0.15 () {54 vars in 962 bytes} [Sun Sep 11 10:46:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:46:34.466192 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ca307e-01fe-4a5b-8014-0c5f3a5de972 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:34.484986 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ca307e-01fe-4a5b-8014-0c5f3a5de972 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:34.487503 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c182c281-17f3-4b80-b7d7-17282ec7f415 service cinder] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:34.492132 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3d064e-7c86-4c6c-b107-5353e44c5487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:34.493155 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc6bbdf-ae83-43ab-b0f9-e1ca55d87fb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:34.493155 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc6bbdf-ae83-43ab-b0f9-e1ca55d87fb5 service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:34.493882 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc6bbdf-ae83-43ab-b0f9-e1ca55d87fb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:34.493882 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc6bbdf-ae83-43ab-b0f9-e1ca55d87fb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:34.496003 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3d064e-7c86-4c6c-b107-5353e44c5487 admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:34.496632 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3d064e-7c86-4c6c-b107-5353e44c5487 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:34.497055 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3d064e-7c86-4c6c-b107-5353e44c5487 admin admin] PATH_INFO: `/v3/projects/160128f88af8425dadd94cf6d46d3ab6/users/464d8c64efbf48678326663e2df51b0f/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:34.505187 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc6bbdf-ae83-43ab-b0f9-e1ca55d87fb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:34.506831 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc6bbdf-ae83-43ab-b0f9-e1ca55d87fb5 service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:34.519582 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 814/1623] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:46:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:46:34.520879 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea153f9c-4bdd-4e65-a12d-12d1d047b513 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:34.521163 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea153f9c-4bdd-4e65-a12d-12d1d047b513 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:34.521440 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea153f9c-4bdd-4e65-a12d-12d1d047b513 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:34.523202 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 815/1624] 10.222.0.15 () {54 vars in 912 bytes} [Sun Sep 11 10:46:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:46:34.524639 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6385395c-0892-4f26-83bb-d9534fa21b4f None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:34.524922 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6385395c-0892-4f26-83bb-d9534fa21b4f None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:34.525194 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6385395c-0892-4f26-83bb-d9534fa21b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:34.535829 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3d064e-7c86-4c6c-b107-5353e44c5487 admin admin] RBAC: Authorizing `identity:create_grant(project_id=160128f88af8425dadd94cf6d46d3ab6, user_id=464d8c64efbf48678326663e2df51b0f, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:34.538975 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3d064e-7c86-4c6c-b107-5353e44c5487 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:34.552977 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-6385395c-0892-4f26-83bb-d9534fa21b4f None None] MFA Rules not processed for user `2bd5356829b6454993bab9bad2f79fed`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:46:34.555224 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-6385395c-0892-4f26-83bb-d9534fa21b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:34.574850 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 810/1625] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:46:34 2022] PUT /identity/v3/projects/160128f88af8425dadd94cf6d46d3ab6/users/464d8c64efbf48678326663e2df51b0f/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:46:34.576197 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f69b880-3ae4-48f2-a2a8-ce033c7111b8 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:34.576458 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f69b880-3ae4-48f2-a2a8-ce033c7111b8 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:34.576767 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f69b880-3ae4-48f2-a2a8-ce033c7111b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:34.577398 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 816/1626] 10.222.0.15 () {58 vars in 1059 bytes} [Sun Sep 11 10:46:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:46:34.579266 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6385395c-0892-4f26-83bb-d9534fa21b4f None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:34.595540 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6385395c-0892-4f26-83bb-d9534fa21b4f None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:34.623948 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-2f69b880-3ae4-48f2-a2a8-ce033c7111b8 None None] MFA Rules not processed for user `2bd5356829b6454993bab9bad2f79fed`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:46:34.637186 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-2f69b880-3ae4-48f2-a2a8-ce033c7111b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:34.686010 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434a430f-9a76-453a-968e-feb16369ba16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:34.687112 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434a430f-9a76-453a-968e-feb16369ba16 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:34.687396 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434a430f-9a76-453a-968e-feb16369ba16 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:34.687765 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434a430f-9a76-453a-968e-feb16369ba16 admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:34.688419 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434a430f-9a76-453a-968e-feb16369ba16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:34.691735 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434a430f-9a76-453a-968e-feb16369ba16 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:34.708140 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 817/1627] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:46:34 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 10:46:34.708140 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434a430f-9a76-453a-968e-feb16369ba16 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:34.765214 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 811/1628] 10.222.0.15 () {58 vars in 1009 bytes} [Sun Sep 11 10:46:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:46:34.775009 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f69b880-3ae4-48f2-a2a8-ce033c7111b8 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:34.788054 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f69b880-3ae4-48f2-a2a8-ce033c7111b8 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:34.797337 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b927700-42ca-46ad-8ae6-2a472dbefde5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:34.798755 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b927700-42ca-46ad-8ae6-2a472dbefde5 admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:34.799109 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b927700-42ca-46ad-8ae6-2a472dbefde5 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:34.799665 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b927700-42ca-46ad-8ae6-2a472dbefde5 admin admin] PATH_INFO: `/v3/projects/160128f88af8425dadd94cf6d46d3ab6/users/464d8c64efbf48678326663e2df51b0f/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:34.803706 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b927700-42ca-46ad-8ae6-2a472dbefde5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=160128f88af8425dadd94cf6d46d3ab6, user_id=464d8c64efbf48678326663e2df51b0f, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:34.809054 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b927700-42ca-46ad-8ae6-2a472dbefde5 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:34.818494 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:46:34.830315 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 812/1629] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:46:34 2022] PUT /identity/v3/projects/160128f88af8425dadd94cf6d46d3ab6/users/464d8c64efbf48678326663e2df51b0f/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:46:34.831238 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b927700-42ca-46ad-8ae6-2a472dbefde5 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:34.892449 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434a430f-9a76-453a-968e-feb16369ba16 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:34.949700 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:46:34.957309 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b927700-42ca-46ad-8ae6-2a472dbefde5 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:34.963690 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2703faa-601b-462c-89a1-a20252785743 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:34.966411 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2703faa-601b-462c-89a1-a20252785743 service cinder] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:34.967655 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2703faa-601b-462c-89a1-a20252785743 service cinder] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:34.968352 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2703faa-601b-462c-89a1-a20252785743 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:35.068712 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eac239-1dcb-4b0d-9b49-39cd80f83041 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bd5356829b6454993bab9bad2f79fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:35.069335 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eac239-1dcb-4b0d-9b49-39cd80f83041 service nova] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:35.069615 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eac239-1dcb-4b0d-9b49-39cd80f83041 service nova] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:35.069887 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eac239-1dcb-4b0d-9b49-39cd80f83041 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:35.091369 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2703faa-601b-462c-89a1-a20252785743 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:35.094450 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2703faa-601b-462c-89a1-a20252785743 service cinder] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:35.140610 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 813/1630] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:46:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:46:35.146712 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef8d7eb-8104-4ac4-8b4f-35790281069e None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:35.148293 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef8d7eb-8104-4ac4-8b4f-35790281069e None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:35.148589 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef8d7eb-8104-4ac4-8b4f-35790281069e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:35.200753 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74eac239-1dcb-4b0d-9b49-39cd80f83041 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:35.205986 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74eac239-1dcb-4b0d-9b49-39cd80f83041 service nova] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:35.232197 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 818/1631] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:46:34 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:46:35.247888 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-4ef8d7eb-8104-4ac4-8b4f-35790281069e None None] MFA Rules not processed for user `464d8c64efbf48678326663e2df51b0f`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:46:35.251695 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-4ef8d7eb-8104-4ac4-8b4f-35790281069e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:35.492046 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 814/1632] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:46:35 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:46:35.497550 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eac239-1dcb-4b0d-9b49-39cd80f83041 service nova] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:35.511347 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eac239-1dcb-4b0d-9b49-39cd80f83041 service nova] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:35.540164 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b993e7c1-a6ef-4933-9e21-c054bd89b1ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:35.543531 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b993e7c1-a6ef-4933-9e21-c054bd89b1ab service cinder] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:35.545607 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b993e7c1-a6ef-4933-9e21-c054bd89b1ab service cinder] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:35.546089 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b993e7c1-a6ef-4933-9e21-c054bd89b1ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:35.573262 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b993e7c1-a6ef-4933-9e21-c054bd89b1ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:35.574939 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b993e7c1-a6ef-4933-9e21-c054bd89b1ab service cinder] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:35.607019 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 819/1633] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:46:35 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:46:35.756820 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272ccdd9-3a94-4dca-9367-49c150e06a7a None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:35.757417 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272ccdd9-3a94-4dca-9367-49c150e06a7a None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:35.757417 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272ccdd9-3a94-4dca-9367-49c150e06a7a None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:35.758845 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 815/1634] 10.222.0.15 () {54 vars in 912 bytes} [Sun Sep 11 10:46:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:46:35.764612 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69a7ffe-c45b-4c74-a526-f2d43df06bc3 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:35.765144 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69a7ffe-c45b-4c74-a526-f2d43df06bc3 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:35.765144 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69a7ffe-c45b-4c74-a526-f2d43df06bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:35.796335 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-e69a7ffe-c45b-4c74-a526-f2d43df06bc3 None None] MFA Rules not processed for user `2bd5356829b6454993bab9bad2f79fed`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:46:35.797692 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-e69a7ffe-c45b-4c74-a526-f2d43df06bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:35.808142 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 820/1635] 10.222.0.15 () {58 vars in 1009 bytes} [Sun Sep 11 10:46:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 10:46:35.855044 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272ccdd9-3a94-4dca-9367-49c150e06a7a None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:35.872902 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272ccdd9-3a94-4dca-9367-49c150e06a7a None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:35.882624 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec94c348-b672-4171-a1ba-56c2948763f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:35.883594 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec94c348-b672-4171-a1ba-56c2948763f8 service cinder] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:35.884928 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec94c348-b672-4171-a1ba-56c2948763f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:35.886389 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec94c348-b672-4171-a1ba-56c2948763f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:35.902891 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec94c348-b672-4171-a1ba-56c2948763f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:35.904624 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec94c348-b672-4171-a1ba-56c2948763f8 service cinder] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:35.923637 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 816/1636] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:46:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:46:36.777302 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69a7ffe-c45b-4c74-a526-f2d43df06bc3 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:36.877281 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69a7ffe-c45b-4c74-a526-f2d43df06bc3 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:36.886221 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3178fc-7d32-447d-acb1-3aaeecbc65fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:36.887028 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3178fc-7d32-447d-acb1-3aaeecbc65fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:36.887361 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3178fc-7d32-447d-acb1-3aaeecbc65fe admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:36.887677 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3178fc-7d32-447d-acb1-3aaeecbc65fe admin admin] PATH_INFO: `/v3/users/7568a199ca0742f09b1e0e02c40bd14a` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:36.889422 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3178fc-7d32-447d-acb1-3aaeecbc65fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=7568a199ca0742f09b1e0e02c40bd14a)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:36.892960 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3178fc-7d32-447d-acb1-3aaeecbc65fe admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:36.973288 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:46:36.976349 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-ab3178fc-7d32-447d-acb1-3aaeecbc65fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7568a199ca0742f09b1e0e02c40bd14a'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:36.987630 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-ab3178fc-7d32-447d-acb1-3aaeecbc65fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7568a199ca0742f09b1e0e02c40bd14a'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:36.988786 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-ab3178fc-7d32-447d-acb1-3aaeecbc65fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7568a199ca0742f09b1e0e02c40bd14a'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:36.989308 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-ab3178fc-7d32-447d-acb1-3aaeecbc65fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7568a199ca0742f09b1e0e02c40bd14a'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:37.005729 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-ab3178fc-7d32-447d-acb1-3aaeecbc65fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7568a199ca0742f09b1e0e02c40bd14a'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:37.027521 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 821/1637] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:46:36 2022] DELETE /identity/v3/users/7568a199ca0742f09b1e0e02c40bd14a => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:46:37.031325 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9a69f-a455-4168-9d01-a29145804c2b None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:37.032196 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9a69f-a455-4168-9d01-a29145804c2b None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:37.032630 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9a69f-a455-4168-9d01-a29145804c2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:37.061094 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-cac9a69f-a455-4168-9d01-a29145804c2b None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:46:37.062566 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-cac9a69f-a455-4168-9d01-a29145804c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:37.067843 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:46:37.155517 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 817/1638] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:46:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 10:46:37.176106 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3178fc-7d32-447d-acb1-3aaeecbc65fe admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:37.177896 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-ab3178fc-7d32-447d-acb1-3aaeecbc65fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:37.292005 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3178fc-7d32-447d-acb1-3aaeecbc65fe admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:37.302930 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a7aaaf-7a5f-4f18-85bb-e6039bdae166 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:37.305080 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a7aaaf-7a5f-4f18-85bb-e6039bdae166 service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:37.306761 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a7aaaf-7a5f-4f18-85bb-e6039bdae166 service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:37.311719 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a7aaaf-7a5f-4f18-85bb-e6039bdae166 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:37.337150 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a7aaaf-7a5f-4f18-85bb-e6039bdae166 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:37.341337 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a7aaaf-7a5f-4f18-85bb-e6039bdae166 service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:37.368842 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 822/1639] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:46:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 10:46:38.469225 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac9a69f-a455-4168-9d01-a29145804c2b None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:38.471233 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-cac9a69f-a455-4168-9d01-a29145804c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:38.500661 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac9a69f-a455-4168-9d01-a29145804c2b None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:38.506453 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765b9cb4-7b84-4106-8ef7-ef1854a0d6e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:38.507115 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765b9cb4-7b84-4106-8ef7-ef1854a0d6e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:38.507399 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765b9cb4-7b84-4106-8ef7-ef1854a0d6e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:38.507702 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765b9cb4-7b84-4106-8ef7-ef1854a0d6e7 admin admin] PATH_INFO: `/v3/projects/7ffde984c6d34c33a44e498ccaa3b668` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:38.508964 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765b9cb4-7b84-4106-8ef7-ef1854a0d6e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ffde984c6d34c33a44e498ccaa3b668)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:38.513435 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765b9cb4-7b84-4106-8ef7-ef1854a0d6e7 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:38.632723 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-765b9cb4-7b84-4106-8ef7-ef1854a0d6e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ffde984c6d34c33a44e498ccaa3b668', 'request_id': 'req-765b9cb4-7b84-4106-8ef7-ef1854a0d6e7'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:38.639673 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-765b9cb4-7b84-4106-8ef7-ef1854a0d6e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ffde984c6d34c33a44e498ccaa3b668', 'request_id': 'req-765b9cb4-7b84-4106-8ef7-ef1854a0d6e7'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:38.659774 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-765b9cb4-7b84-4106-8ef7-ef1854a0d6e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ffde984c6d34c33a44e498ccaa3b668', 'request_id': 'req-765b9cb4-7b84-4106-8ef7-ef1854a0d6e7'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:38.680347 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-765b9cb4-7b84-4106-8ef7-ef1854a0d6e7 admin admin] The token cache is being invalidate because project 7ffde984c6d34c33a44e498ccaa3b668 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90653) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:46:38.681539 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-765b9cb4-7b84-4106-8ef7-ef1854a0d6e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:38.682520 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-765b9cb4-7b84-4106-8ef7-ef1854a0d6e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:38.697819 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a7aaaf-7a5f-4f18-85bb-e6039bdae166 service neutron] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:38.715605 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 818/1640] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:46:38 2022] DELETE /identity/v3/projects/7ffde984c6d34c33a44e498ccaa3b668 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:46:38.724827 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a7aaaf-7a5f-4f18-85bb-e6039bdae166 service neutron] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:38.848381 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a37b679-b66f-4703-8d23-76a90f746a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:38.850462 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a37b679-b66f-4703-8d23-76a90f746a87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:38.851729 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a37b679-b66f-4703-8d23-76a90f746a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:38.852246 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a37b679-b66f-4703-8d23-76a90f746a87 admin admin] PATH_INFO: `/v3/users/464d8c64efbf48678326663e2df51b0f` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:38.854285 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a37b679-b66f-4703-8d23-76a90f746a87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=464d8c64efbf48678326663e2df51b0f)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:38.858278 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a37b679-b66f-4703-8d23-76a90f746a87 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:38.932080 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-1a37b679-b66f-4703-8d23-76a90f746a87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '464d8c64efbf48678326663e2df51b0f'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:38.946786 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-1a37b679-b66f-4703-8d23-76a90f746a87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '464d8c64efbf48678326663e2df51b0f'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:38.949208 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-1a37b679-b66f-4703-8d23-76a90f746a87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '464d8c64efbf48678326663e2df51b0f'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:38.950132 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-1a37b679-b66f-4703-8d23-76a90f746a87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '464d8c64efbf48678326663e2df51b0f'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:38.964165 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:46:38.967271 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-1a37b679-b66f-4703-8d23-76a90f746a87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '464d8c64efbf48678326663e2df51b0f'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:38.989388 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 823/1641] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:46:38 2022] DELETE /identity/v3/users/464d8c64efbf48678326663e2df51b0f => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:46:40.919066 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765b9cb4-7b84-4106-8ef7-ef1854a0d6e7 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:40.920791 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-765b9cb4-7b84-4106-8ef7-ef1854a0d6e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:40.968408 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765b9cb4-7b84-4106-8ef7-ef1854a0d6e7 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:41.156846 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b47118-49e4-450d-92f2-17e6a8b5263e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:41.157023 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b47118-49e4-450d-92f2-17e6a8b5263e admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:41.157252 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b47118-49e4-450d-92f2-17e6a8b5263e admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:41.157449 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b47118-49e4-450d-92f2-17e6a8b5263e admin admin] PATH_INFO: `/v3/projects/160128f88af8425dadd94cf6d46d3ab6` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:41.164353 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b47118-49e4-450d-92f2-17e6a8b5263e admin admin] RBAC: Authorizing `identity:delete_project(project_id=160128f88af8425dadd94cf6d46d3ab6)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:41.166333 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b47118-49e4-450d-92f2-17e6a8b5263e admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:41.248717 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:46:41.277841 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-d2b47118-49e4-450d-92f2-17e6a8b5263e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '160128f88af8425dadd94cf6d46d3ab6', 'request_id': 'req-d2b47118-49e4-450d-92f2-17e6a8b5263e'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:41.297517 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-d2b47118-49e4-450d-92f2-17e6a8b5263e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '160128f88af8425dadd94cf6d46d3ab6', 'request_id': 'req-d2b47118-49e4-450d-92f2-17e6a8b5263e'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:41.313852 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-d2b47118-49e4-450d-92f2-17e6a8b5263e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '160128f88af8425dadd94cf6d46d3ab6', 'request_id': 'req-d2b47118-49e4-450d-92f2-17e6a8b5263e'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:41.336551 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-d2b47118-49e4-450d-92f2-17e6a8b5263e admin admin] The token cache is being invalidate because project 160128f88af8425dadd94cf6d46d3ab6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90653) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:46:41.338660 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-d2b47118-49e4-450d-92f2-17e6a8b5263e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:41.340512 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-d2b47118-49e4-450d-92f2-17e6a8b5263e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:41.343787 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 819/1642] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:46:40 2022] DELETE /identity/v3/projects/160128f88af8425dadd94cf6d46d3ab6 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:46:41.349243 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f584b9-6dd3-43a2-99d5-6db713ffd1e2 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:41.350311 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f584b9-6dd3-43a2-99d5-6db713ffd1e2 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:41.350889 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f584b9-6dd3-43a2-99d5-6db713ffd1e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:41.412740 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-47f584b9-6dd3-43a2-99d5-6db713ffd1e2 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:46:41.414003 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-47f584b9-6dd3-43a2-99d5-6db713ffd1e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:41.575801 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 824/1643] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:46:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:46:41.593573 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3193863f-6c9b-414a-abcc-88f8c5b10618 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:41.594163 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3193863f-6c9b-414a-abcc-88f8c5b10618 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:41.594532 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3193863f-6c9b-414a-abcc-88f8c5b10618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:41.658349 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-3193863f-6c9b-414a-abcc-88f8c5b10618 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:46:41.661515 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-3193863f-6c9b-414a-abcc-88f8c5b10618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:41.687652 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 820/1644] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:46:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:46:41.690863 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f584b9-6dd3-43a2-99d5-6db713ffd1e2 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:41.703973 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f584b9-6dd3-43a2-99d5-6db713ffd1e2 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:41.720098 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875372e0-5b54-408f-bea5-4b6b8454583a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:41.720741 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875372e0-5b54-408f-bea5-4b6b8454583a admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:41.721024 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875372e0-5b54-408f-bea5-4b6b8454583a admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:41.721302 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875372e0-5b54-408f-bea5-4b6b8454583a admin admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:41.724121 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875372e0-5b54-408f-bea5-4b6b8454583a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:46:41.724476 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875372e0-5b54-408f-bea5-4b6b8454583a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:41.728936 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875372e0-5b54-408f-bea5-4b6b8454583a admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:41.769977 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 825/1645] 10.222.0.15 () {58 vars in 1153 bytes} [Sun Sep 11 10:46:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:46:41.774281 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3193863f-6c9b-414a-abcc-88f8c5b10618 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:41.789667 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3193863f-6c9b-414a-abcc-88f8c5b10618 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:41.804396 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42a178c-8cde-4c89-a052-ec629dc2de45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:41.806704 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42a178c-8cde-4c89-a052-ec629dc2de45 admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:41.807144 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42a178c-8cde-4c89-a052-ec629dc2de45 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:41.808988 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42a178c-8cde-4c89-a052-ec629dc2de45 admin admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:41.811312 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42a178c-8cde-4c89-a052-ec629dc2de45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:41.815884 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42a178c-8cde-4c89-a052-ec629dc2de45 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:41.881774 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 821/1646] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:46:41 2022] POST /identity/v3/projects => generated 365 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:46:41.884562 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875372e0-5b54-408f-bea5-4b6b8454583a admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:41.903959 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875372e0-5b54-408f-bea5-4b6b8454583a admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:42.070808 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5df525-f108-4728-8330-937a291c3889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:42.071485 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5df525-f108-4728-8330-937a291c3889 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:42.071774 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5df525-f108-4728-8330-937a291c3889 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:42.072045 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5df525-f108-4728-8330-937a291c3889 admin admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:42.072697 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5df525-f108-4728-8330-937a291c3889 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:42.074963 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5df525-f108-4728-8330-937a291c3889 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:42.160299 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 826/1647] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:46:41 2022] POST /identity/v3/users => generated 283 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 10:46:42.166563 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42a178c-8cde-4c89-a052-ec629dc2de45 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:42.183517 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42a178c-8cde-4c89-a052-ec629dc2de45 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:42.188860 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446bb179-a0af-4565-8629-6793c20d2bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:42.189765 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446bb179-a0af-4565-8629-6793c20d2bd5 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:42.190238 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446bb179-a0af-4565-8629-6793c20d2bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:42.192529 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446bb179-a0af-4565-8629-6793c20d2bd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:42.193306 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446bb179-a0af-4565-8629-6793c20d2bd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:42.195303 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446bb179-a0af-4565-8629-6793c20d2bd5 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:42.210607 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 822/1648] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:46:42 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:46:42.211946 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5df525-f108-4728-8330-937a291c3889 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:42.229451 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5df525-f108-4728-8330-937a291c3889 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:42.252627 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23271673-b4c8-40d0-9c4e-8b3f62c7658a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:42.253653 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23271673-b4c8-40d0-9c4e-8b3f62c7658a admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:42.254509 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23271673-b4c8-40d0-9c4e-8b3f62c7658a admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:42.268596 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23271673-b4c8-40d0-9c4e-8b3f62c7658a admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:42.270541 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23271673-b4c8-40d0-9c4e-8b3f62c7658a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:42.273841 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23271673-b4c8-40d0-9c4e-8b3f62c7658a admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:42.316162 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 827/1649] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:46:42 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:46:42.318166 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446bb179-a0af-4565-8629-6793c20d2bd5 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:42.349852 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446bb179-a0af-4565-8629-6793c20d2bd5 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:42.378222 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a0293d-74d5-4980-8064-e169ed5148f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:42.380888 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a0293d-74d5-4980-8064-e169ed5148f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:42.381650 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a0293d-74d5-4980-8064-e169ed5148f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:42.382407 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a0293d-74d5-4980-8064-e169ed5148f5 admin admin] PATH_INFO: `/v3/projects/7745546a1c6d481a9d0bf4984a536507/users/f71ad961be00412a88e6f2bf5757ae40/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:42.466987 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a0293d-74d5-4980-8064-e169ed5148f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7745546a1c6d481a9d0bf4984a536507, user_id=f71ad961be00412a88e6f2bf5757ae40, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:42.472823 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a0293d-74d5-4980-8064-e169ed5148f5 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:42.517427 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 823/1650] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:46:42 2022] PUT /identity/v3/projects/7745546a1c6d481a9d0bf4984a536507/users/f71ad961be00412a88e6f2bf5757ae40/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:46:42.529716 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd500eb-31a5-4cd7-82ba-3802efec5cf4 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:42.529716 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd500eb-31a5-4cd7-82ba-3802efec5cf4 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:42.530340 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd500eb-31a5-4cd7-82ba-3802efec5cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:42.670764 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-1dd500eb-31a5-4cd7-82ba-3802efec5cf4 None None] MFA Rules not processed for user `f71ad961be00412a88e6f2bf5757ae40`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:46:42.677330 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-1dd500eb-31a5-4cd7-82ba-3802efec5cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:42.888577 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:46:42.995208 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 828/1651] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:46:42 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 468 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:46:42.999721 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a0293d-74d5-4980-8064-e169ed5148f5 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:43.094458 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af10d44a-d0b5-48c9-af65-7594e1dc4f6b None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:43.094781 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af10d44a-d0b5-48c9-af65-7594e1dc4f6b None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:43.095022 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af10d44a-d0b5-48c9-af65-7594e1dc4f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:43.129566 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a0293d-74d5-4980-8064-e169ed5148f5 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:43.160861 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6de06f-4f8d-4313-aa68-1d484b51cfec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:43.162441 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6de06f-4f8d-4313-aa68-1d484b51cfec admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:43.163824 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6de06f-4f8d-4313-aa68-1d484b51cfec admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:43.164120 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6de06f-4f8d-4313-aa68-1d484b51cfec admin admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:43.165843 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6de06f-4f8d-4313-aa68-1d484b51cfec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:43.171063 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-af10d44a-d0b5-48c9-af65-7594e1dc4f6b None None] MFA Rules not processed for user `8d0af2b5f354439d92bfa317a0baefd5`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:46:43.171837 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-af10d44a-d0b5-48c9-af65-7594e1dc4f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:43.172142 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6de06f-4f8d-4313-aa68-1d484b51cfec admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:43.266943 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 824/1652] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:46:42 2022] POST /identity/v3/projects => generated 365 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:46:43.271410 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6de06f-4f8d-4313-aa68-1d484b51cfec admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:43.315469 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 829/1653] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:46:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 10:46:43.330842 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af10d44a-d0b5-48c9-af65-7594e1dc4f6b None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:43.333132 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-af10d44a-d0b5-48c9-af65-7594e1dc4f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:43.371867 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:46:43.405022 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6de06f-4f8d-4313-aa68-1d484b51cfec admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:43.428214 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b08c1f-433c-43cb-9961-66c2ef9fae96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:43.429229 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b08c1f-433c-43cb-9961-66c2ef9fae96 admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:43.429715 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b08c1f-433c-43cb-9961-66c2ef9fae96 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:43.430182 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b08c1f-433c-43cb-9961-66c2ef9fae96 admin admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:43.431044 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b08c1f-433c-43cb-9961-66c2ef9fae96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:43.433127 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b08c1f-433c-43cb-9961-66c2ef9fae96 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:43.465578 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 825/1654] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:46:43 2022] POST /identity/v3/users => generated 289 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:46:43.469446 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b08c1f-433c-43cb-9961-66c2ef9fae96 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:43.488753 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af10d44a-d0b5-48c9-af65-7594e1dc4f6b None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:43.492005 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b08c1f-433c-43cb-9961-66c2ef9fae96 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:43.514775 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe7a2fd-f796-41a7-8e86-77b843423d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:43.515402 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe7a2fd-f796-41a7-8e86-77b843423d24 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:43.515710 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe7a2fd-f796-41a7-8e86-77b843423d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:43.515984 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe7a2fd-f796-41a7-8e86-77b843423d24 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:43.516631 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe7a2fd-f796-41a7-8e86-77b843423d24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:43.518237 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe7a2fd-f796-41a7-8e86-77b843423d24 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:43.535482 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 826/1655] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:46:43 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:46:43.536394 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe7a2fd-f796-41a7-8e86-77b843423d24 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:43.548650 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe7a2fd-f796-41a7-8e86-77b843423d24 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:43.555031 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f957476d-8a55-45d0-9b12-aaf3bb36983e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:43.556020 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f957476d-8a55-45d0-9b12-aaf3bb36983e admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:43.556452 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f957476d-8a55-45d0-9b12-aaf3bb36983e admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:43.556735 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f957476d-8a55-45d0-9b12-aaf3bb36983e admin admin] PATH_INFO: `/v3/projects/146ff83c68434a60a1c6b655f5d9adfa/users/9ab443b09fe54923ac8cc15609bdedfc/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:43.579116 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f957476d-8a55-45d0-9b12-aaf3bb36983e admin admin] RBAC: Authorizing `identity:create_grant(project_id=146ff83c68434a60a1c6b655f5d9adfa, user_id=9ab443b09fe54923ac8cc15609bdedfc, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:43.588804 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f957476d-8a55-45d0-9b12-aaf3bb36983e admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:43.619749 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 827/1656] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:46:43 2022] PUT /identity/v3/projects/146ff83c68434a60a1c6b655f5d9adfa/users/9ab443b09fe54923ac8cc15609bdedfc/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:46:43.625847 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f957476d-8a55-45d0-9b12-aaf3bb36983e admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:43.627328 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7326b11-5bb2-4e91-a15d-a8e3d5bb0075 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bd5356829b6454993bab9bad2f79fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:43.628019 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7326b11-5bb2-4e91-a15d-a8e3d5bb0075 service nova] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:43.628481 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7326b11-5bb2-4e91-a15d-a8e3d5bb0075 service nova] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:43.628803 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7326b11-5bb2-4e91-a15d-a8e3d5bb0075 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:43.643259 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:46:43.764874 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f957476d-8a55-45d0-9b12-aaf3bb36983e admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:43.772400 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c970ac02-79ac-4494-93e1-56cd6e23e6ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:43.773153 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c970ac02-79ac-4494-93e1-56cd6e23e6ea admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:43.773903 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c970ac02-79ac-4494-93e1-56cd6e23e6ea admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:43.774279 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c970ac02-79ac-4494-93e1-56cd6e23e6ea admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:43.775052 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c970ac02-79ac-4494-93e1-56cd6e23e6ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:43.777096 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c970ac02-79ac-4494-93e1-56cd6e23e6ea admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:43.792879 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 828/1657] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:46:43 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:46:43.796539 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c970ac02-79ac-4494-93e1-56cd6e23e6ea admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:43.813545 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c970ac02-79ac-4494-93e1-56cd6e23e6ea admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:43.816534 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7326b11-5bb2-4e91-a15d-a8e3d5bb0075 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:43.818338 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7326b11-5bb2-4e91-a15d-a8e3d5bb0075 service nova] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:43.820407 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1137d3-7e80-4d1c-8fd5-19f4000001f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:43.821330 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1137d3-7e80-4d1c-8fd5-19f4000001f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:43.821743 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1137d3-7e80-4d1c-8fd5-19f4000001f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:43.822093 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1137d3-7e80-4d1c-8fd5-19f4000001f1 admin admin] PATH_INFO: `/v3/projects/146ff83c68434a60a1c6b655f5d9adfa/users/9ab443b09fe54923ac8cc15609bdedfc/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:43.825073 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1137d3-7e80-4d1c-8fd5-19f4000001f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=146ff83c68434a60a1c6b655f5d9adfa, user_id=9ab443b09fe54923ac8cc15609bdedfc, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:43.828418 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1137d3-7e80-4d1c-8fd5-19f4000001f1 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:43.849814 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 829/1658] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:46:43 2022] PUT /identity/v3/projects/146ff83c68434a60a1c6b655f5d9adfa/users/9ab443b09fe54923ac8cc15609bdedfc/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:46:43.852424 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 830/1659] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:46:43 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:46:43.861813 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fd77c7-1c71-4dc3-ac61-f1757f799e71 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:43.862175 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fd77c7-1c71-4dc3-ac61-f1757f799e71 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:43.862444 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fd77c7-1c71-4dc3-ac61-f1757f799e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:44.028173 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-40fd77c7-1c71-4dc3-ac61-f1757f799e71 None None] MFA Rules not processed for user `9ab443b09fe54923ac8cc15609bdedfc`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:46:44.039677 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-40fd77c7-1c71-4dc3-ac61-f1757f799e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:44.215167 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 830/1660] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:46:43 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:46:44.228711 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7326b11-5bb2-4e91-a15d-a8e3d5bb0075 service nova] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:44.230347 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-f7326b11-5bb2-4e91-a15d-a8e3d5bb0075 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:44.405825 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7326b11-5bb2-4e91-a15d-a8e3d5bb0075 service nova] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:44.582299 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53281445-cb7f-4c98-86b5-54b252e0b126 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:44.593620 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53281445-cb7f-4c98-86b5-54b252e0b126 service cinder] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:44.593894 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53281445-cb7f-4c98-86b5-54b252e0b126 service cinder] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:44.594237 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53281445-cb7f-4c98-86b5-54b252e0b126 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:44.625400 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53281445-cb7f-4c98-86b5-54b252e0b126 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:44.627190 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53281445-cb7f-4c98-86b5-54b252e0b126 service cinder] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:44.641621 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 831/1661] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:46:44 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:46:44.817396 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fd77c7-1c71-4dc3-ac61-f1757f799e71 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:44.832508 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fd77c7-1c71-4dc3-ac61-f1757f799e71 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:44.844247 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b1e247-c43c-4651-8b74-9a3d868d84e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:44.845508 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b1e247-c43c-4651-8b74-9a3d868d84e8 service cinder] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:44.845992 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b1e247-c43c-4651-8b74-9a3d868d84e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:44.846464 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b1e247-c43c-4651-8b74-9a3d868d84e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:44.859733 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:46:44.963027 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b1e247-c43c-4651-8b74-9a3d868d84e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:44.966243 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b1e247-c43c-4651-8b74-9a3d868d84e8 service cinder] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:44.996908 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 831/1662] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:46:44 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:46:45.533868 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53281445-cb7f-4c98-86b5-54b252e0b126 service cinder] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:45.535404 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-53281445-cb7f-4c98-86b5-54b252e0b126 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:45.758373 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53281445-cb7f-4c98-86b5-54b252e0b126 service cinder] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:45.781011 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406aef18-24a1-422e-ada9-e84d4dc2f711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:45.781687 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406aef18-24a1-422e-ada9-e84d4dc2f711 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:45.781972 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406aef18-24a1-422e-ada9-e84d4dc2f711 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:45.782380 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406aef18-24a1-422e-ada9-e84d4dc2f711 admin admin] PATH_INFO: `/v3/users/1eb7a55cacf3400cbbdead369f9f65e7` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:45.789697 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406aef18-24a1-422e-ada9-e84d4dc2f711 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1eb7a55cacf3400cbbdead369f9f65e7)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:45.791644 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406aef18-24a1-422e-ada9-e84d4dc2f711 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:45.820355 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:46:45.959934 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-406aef18-24a1-422e-ada9-e84d4dc2f711 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1eb7a55cacf3400cbbdead369f9f65e7'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:45.978883 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-406aef18-24a1-422e-ada9-e84d4dc2f711 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1eb7a55cacf3400cbbdead369f9f65e7'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:45.980026 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-406aef18-24a1-422e-ada9-e84d4dc2f711 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1eb7a55cacf3400cbbdead369f9f65e7'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:45.980780 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-406aef18-24a1-422e-ada9-e84d4dc2f711 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1eb7a55cacf3400cbbdead369f9f65e7'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:46.015254 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-406aef18-24a1-422e-ada9-e84d4dc2f711 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1eb7a55cacf3400cbbdead369f9f65e7'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:46.047873 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 832/1663] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:46:45 2022] DELETE /identity/v3/users/1eb7a55cacf3400cbbdead369f9f65e7 => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:46:46.049431 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd9e81-434d-4207-ab1b-00ca72f8f12c None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:46.049866 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd9e81-434d-4207-ab1b-00ca72f8f12c None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:46.050068 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd9e81-434d-4207-ab1b-00ca72f8f12c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:46.105405 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-33dd9e81-434d-4207-ab1b-00ca72f8f12c None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:46:46.109421 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-33dd9e81-434d-4207-ab1b-00ca72f8f12c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:46.281539 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 832/1664] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:46:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:46:46.416630 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406aef18-24a1-422e-ada9-e84d4dc2f711 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:46.418560 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-406aef18-24a1-422e-ada9-e84d4dc2f711 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:46.611890 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406aef18-24a1-422e-ada9-e84d4dc2f711 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:46.619102 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fda74d0-dc82-4d99-ad27-9bd26ca12e6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:46.622806 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fda74d0-dc82-4d99-ad27-9bd26ca12e6e service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:46.623821 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fda74d0-dc82-4d99-ad27-9bd26ca12e6e service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:46.624498 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fda74d0-dc82-4d99-ad27-9bd26ca12e6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:46.652941 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fda74d0-dc82-4d99-ad27-9bd26ca12e6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:46.656613 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fda74d0-dc82-4d99-ad27-9bd26ca12e6e service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:46.694288 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 833/1665] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:46:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:46:47.342025 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb38952-010f-407e-aa4c-48d4ef806337 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:47.342840 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb38952-010f-407e-aa4c-48d4ef806337 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:47.343254 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb38952-010f-407e-aa4c-48d4ef806337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:47.408351 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-8bb38952-010f-407e-aa4c-48d4ef806337 None None] MFA Rules not processed for user `2238a62cbf1846999b31ea9434b67743`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:46:47.411529 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-8bb38952-010f-407e-aa4c-48d4ef806337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:47.650383 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 833/1666] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:46:47 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:46:47.665780 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fda74d0-dc82-4d99-ad27-9bd26ca12e6e service neutron] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:47.667687 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-6fda74d0-dc82-4d99-ad27-9bd26ca12e6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:47.835867 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb38952-010f-407e-aa4c-48d4ef806337 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:47.837411 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-8bb38952-010f-407e-aa4c-48d4ef806337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:47.891319 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb38952-010f-407e-aa4c-48d4ef806337 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:47.908450 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523e6934-3563-46c0-be07-f3c7ee8237da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:47.909118 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e6934-3563-46c0-be07-f3c7ee8237da admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:47.909936 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e6934-3563-46c0-be07-f3c7ee8237da admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:47.910212 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e6934-3563-46c0-be07-f3c7ee8237da admin admin] PATH_INFO: `/v3/projects/22d7f01aefde4648ac6332e67d622214` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:47.916653 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fda74d0-dc82-4d99-ad27-9bd26ca12e6e service neutron] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:47.917883 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523e6934-3563-46c0-be07-f3c7ee8237da admin admin] RBAC: Authorizing `identity:delete_project(project_id=22d7f01aefde4648ac6332e67d622214)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:47.919888 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523e6934-3563-46c0-be07-f3c7ee8237da admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:47.935846 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac385d34-1fe3-4d78-8e33-645cec898e03 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bd5356829b6454993bab9bad2f79fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:47.936760 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac385d34-1fe3-4d78-8e33-645cec898e03 service nova] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:47.937193 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac385d34-1fe3-4d78-8e33-645cec898e03 service nova] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:47.937749 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac385d34-1fe3-4d78-8e33-645cec898e03 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:47.973605 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac385d34-1fe3-4d78-8e33-645cec898e03 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:47.975688 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac385d34-1fe3-4d78-8e33-645cec898e03 service nova] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:47.998008 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 834/1667] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:46:47 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:46:48.034008 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:46:48.087820 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-523e6934-3563-46c0-be07-f3c7ee8237da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22d7f01aefde4648ac6332e67d622214', 'request_id': 'req-523e6934-3563-46c0-be07-f3c7ee8237da'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:48.122867 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-523e6934-3563-46c0-be07-f3c7ee8237da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22d7f01aefde4648ac6332e67d622214', 'request_id': 'req-523e6934-3563-46c0-be07-f3c7ee8237da'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:48.138502 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-523e6934-3563-46c0-be07-f3c7ee8237da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22d7f01aefde4648ac6332e67d622214', 'request_id': 'req-523e6934-3563-46c0-be07-f3c7ee8237da'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:48.177442 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-523e6934-3563-46c0-be07-f3c7ee8237da admin admin] The token cache is being invalidate because project 22d7f01aefde4648ac6332e67d622214 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90653) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:46:48.177752 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-523e6934-3563-46c0-be07-f3c7ee8237da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:48.178439 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-523e6934-3563-46c0-be07-f3c7ee8237da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:48.184107 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 834/1668] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:46:47 2022] DELETE /identity/v3/projects/22d7f01aefde4648ac6332e67d622214 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 10:46:48.189364 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7d73ac-1a00-4e46-aab7-54ea790c6b07 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:48.189653 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7d73ac-1a00-4e46-aab7-54ea790c6b07 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:48.191031 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7d73ac-1a00-4e46-aab7-54ea790c6b07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:48.260621 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-ee7d73ac-1a00-4e46-aab7-54ea790c6b07 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:46:48.266124 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-ee7d73ac-1a00-4e46-aab7-54ea790c6b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:48.300707 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:46:48.480550 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523e6934-3563-46c0-be07-f3c7ee8237da admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:48.483313 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-523e6934-3563-46c0-be07-f3c7ee8237da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:48.524695 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 835/1669] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:46:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 10:46:48.532669 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1ad03b-1e2f-41e7-afc8-f097df72e8ed None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:48.533194 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1ad03b-1e2f-41e7-afc8-f097df72e8ed None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:48.533680 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1ad03b-1e2f-41e7-afc8-f097df72e8ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:48.600774 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-bb1ad03b-1e2f-41e7-afc8-f097df72e8ed None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:46:48.602527 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-bb1ad03b-1e2f-41e7-afc8-f097df72e8ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:48.615659 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 836/1670] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:46:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 10:46:48.619596 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1ad03b-1e2f-41e7-afc8-f097df72e8ed None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:48.644467 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523e6934-3563-46c0-be07-f3c7ee8237da admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:48.645701 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1ad03b-1e2f-41e7-afc8-f097df72e8ed None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:48.650343 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b17e5c-755a-49f8-b49f-73aff1b4012a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:48.651170 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b17e5c-755a-49f8-b49f-73aff1b4012a admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:48.651857 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b17e5c-755a-49f8-b49f-73aff1b4012a admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:48.652230 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b17e5c-755a-49f8-b49f-73aff1b4012a admin admin] PATH_INFO: `/v3/users/2540f47512f74979948ba4ffa3f0415f` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:48.658028 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b17e5c-755a-49f8-b49f-73aff1b4012a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2540f47512f74979948ba4ffa3f0415f)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:48.658396 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37e2f8d-5518-487e-aa66-374d04d14aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:48.659060 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: WARNING py.warnings [None req-e7b17e5c-755a-49f8-b49f-73aff1b4012a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 11 10:46:48.659060 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: warnings.warn(msg) Sep 11 10:46:48.659060 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: Sep 11 10:46:48.659842 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37e2f8d-5518-487e-aa66-374d04d14aa1 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:48.661082 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b17e5c-755a-49f8-b49f-73aff1b4012a admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:48.661767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37e2f8d-5518-487e-aa66-374d04d14aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:48.662125 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37e2f8d-5518-487e-aa66-374d04d14aa1 admin admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:48.662805 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37e2f8d-5518-487e-aa66-374d04d14aa1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:46:48.664233 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37e2f8d-5518-487e-aa66-374d04d14aa1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:48.670242 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37e2f8d-5518-487e-aa66-374d04d14aa1 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:48.722755 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 837/1671] 10.222.0.15 () {58 vars in 1153 bytes} [Sun Sep 11 10:46:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:46:48.723734 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37e2f8d-5518-487e-aa66-374d04d14aa1 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:48.747113 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37e2f8d-5518-487e-aa66-374d04d14aa1 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:48.762340 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d50f60-9e80-4c72-a35d-2f45527313fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:48.762992 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d50f60-9e80-4c72-a35d-2f45527313fe admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:48.763324 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d50f60-9e80-4c72-a35d-2f45527313fe admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:48.763630 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d50f60-9e80-4c72-a35d-2f45527313fe admin admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:48.764303 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d50f60-9e80-4c72-a35d-2f45527313fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:48.767158 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d50f60-9e80-4c72-a35d-2f45527313fe admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:48.847741 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 838/1672] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:46:48 2022] POST /identity/v3/projects => generated 379 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:46:48.850604 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d50f60-9e80-4c72-a35d-2f45527313fe admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:48.863516 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-e7b17e5c-755a-49f8-b49f-73aff1b4012a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2540f47512f74979948ba4ffa3f0415f'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:48.882219 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-e7b17e5c-755a-49f8-b49f-73aff1b4012a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2540f47512f74979948ba4ffa3f0415f'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:48.883088 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-e7b17e5c-755a-49f8-b49f-73aff1b4012a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2540f47512f74979948ba4ffa3f0415f'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:48.883756 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-e7b17e5c-755a-49f8-b49f-73aff1b4012a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2540f47512f74979948ba4ffa3f0415f'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:48.921312 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-e7b17e5c-755a-49f8-b49f-73aff1b4012a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2540f47512f74979948ba4ffa3f0415f'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:48.956120 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d50f60-9e80-4c72-a35d-2f45527313fe admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:48.963357 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29bcf5a-02fd-4d65-8a52-0a50b31feb4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:48.964281 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29bcf5a-02fd-4d65-8a52-0a50b31feb4e admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:48.965385 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29bcf5a-02fd-4d65-8a52-0a50b31feb4e admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:48.965986 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29bcf5a-02fd-4d65-8a52-0a50b31feb4e admin admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:48.966691 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29bcf5a-02fd-4d65-8a52-0a50b31feb4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:48.969555 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29bcf5a-02fd-4d65-8a52-0a50b31feb4e admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:48.995797 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 835/1673] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:46:48 2022] DELETE /identity/v3/users/2540f47512f74979948ba4ffa3f0415f => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:46:49.000552 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c573a8e5-66f1-49b9-af3f-6a375300fcd1 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:49.000864 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c573a8e5-66f1-49b9-af3f-6a375300fcd1 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:49.001165 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c573a8e5-66f1-49b9-af3f-6a375300fcd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:49.010055 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 839/1674] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:46:48 2022] POST /identity/v3/users => generated 290 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:46:49.012233 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29bcf5a-02fd-4d65-8a52-0a50b31feb4e admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:49.021426 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-c29bcf5a-02fd-4d65-8a52-0a50b31feb4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:49.071702 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29bcf5a-02fd-4d65-8a52-0a50b31feb4e admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:49.072179 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-c573a8e5-66f1-49b9-af3f-6a375300fcd1 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:46:49.078096 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-c573a8e5-66f1-49b9-af3f-6a375300fcd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:49.220343 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:46:49.315204 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6b0f21-293e-4606-ab5d-c6ec771380a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:49.315975 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b0f21-293e-4606-ab5d-c6ec771380a7 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:49.319174 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b0f21-293e-4606-ab5d-c6ec771380a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:49.320624 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b0f21-293e-4606-ab5d-c6ec771380a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:49.324426 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6b0f21-293e-4606-ab5d-c6ec771380a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:49.325978 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 836/1675] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:46:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:46:49.326906 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c573a8e5-66f1-49b9-af3f-6a375300fcd1 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:49.328255 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6b0f21-293e-4606-ab5d-c6ec771380a7 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:49.330234 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-c573a8e5-66f1-49b9-af3f-6a375300fcd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:49.336065 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:46:49.374865 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 840/1676] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:46:49 2022] GET /identity/v3/roles => generated 1420 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:46:49.375851 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6b0f21-293e-4606-ab5d-c6ec771380a7 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:49.390214 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c573a8e5-66f1-49b9-af3f-6a375300fcd1 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:49.413193 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a19c42-82ed-4047-b287-dd3eb16471e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:49.415886 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a19c42-82ed-4047-b287-dd3eb16471e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:49.416241 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a19c42-82ed-4047-b287-dd3eb16471e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:49.416590 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a19c42-82ed-4047-b287-dd3eb16471e4 admin admin] PATH_INFO: `/v3/users/f71ad961be00412a88e6f2bf5757ae40` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:49.425953 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a19c42-82ed-4047-b287-dd3eb16471e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f71ad961be00412a88e6f2bf5757ae40)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:49.427940 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6b0f21-293e-4606-ab5d-c6ec771380a7 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:49.437587 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a19c42-82ed-4047-b287-dd3eb16471e4 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:49.450228 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3693a65-55ba-430b-8d54-6ff61d03838b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:49.450860 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3693a65-55ba-430b-8d54-6ff61d03838b admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:49.451851 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3693a65-55ba-430b-8d54-6ff61d03838b admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:49.452132 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3693a65-55ba-430b-8d54-6ff61d03838b admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:49.455738 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3693a65-55ba-430b-8d54-6ff61d03838b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:49.457788 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3693a65-55ba-430b-8d54-6ff61d03838b admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:49.560357 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 841/1677] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:46:49 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:46:49.561600 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3693a65-55ba-430b-8d54-6ff61d03838b admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:49.573896 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-d3693a65-55ba-430b-8d54-6ff61d03838b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:49.660523 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-01a19c42-82ed-4047-b287-dd3eb16471e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f71ad961be00412a88e6f2bf5757ae40'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:49.679010 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-01a19c42-82ed-4047-b287-dd3eb16471e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f71ad961be00412a88e6f2bf5757ae40'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:49.679637 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-01a19c42-82ed-4047-b287-dd3eb16471e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f71ad961be00412a88e6f2bf5757ae40'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:49.680406 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-01a19c42-82ed-4047-b287-dd3eb16471e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f71ad961be00412a88e6f2bf5757ae40'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:49.728242 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-01a19c42-82ed-4047-b287-dd3eb16471e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f71ad961be00412a88e6f2bf5757ae40'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:49.802781 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 837/1678] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:46:49 2022] DELETE /identity/v3/users/f71ad961be00412a88e6f2bf5757ae40 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:46:49.804072 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a19c42-82ed-4047-b287-dd3eb16471e4 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:49.819192 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-01a19c42-82ed-4047-b287-dd3eb16471e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:49.823988 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3693a65-55ba-430b-8d54-6ff61d03838b admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:49.951374 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a19c42-82ed-4047-b287-dd3eb16471e4 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:49.957949 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b38014-9b27-47d2-93d3-15dfcdd0f3cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:49.958968 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b38014-9b27-47d2-93d3-15dfcdd0f3cb service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:49.959259 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b38014-9b27-47d2-93d3-15dfcdd0f3cb service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:49.959562 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b38014-9b27-47d2-93d3-15dfcdd0f3cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:49.964858 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-12b38014-9b27-47d2-93d3-15dfcdd0f3cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:49.969824 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7fe12b-ac95-44a2-ad78-59b9fb854c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:49.971104 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7fe12b-ac95-44a2-ad78-59b9fb854c32 admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:49.971859 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7fe12b-ac95-44a2-ad78-59b9fb854c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:49.972675 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7fe12b-ac95-44a2-ad78-59b9fb854c32 admin admin] PATH_INFO: `/v3/projects/a37be821dec241f29ddba0a7b68f2801/users/fc7d9d0572e246eea84c626440f66706/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:49.994629 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b38014-9b27-47d2-93d3-15dfcdd0f3cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:49.996129 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b38014-9b27-47d2-93d3-15dfcdd0f3cb service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:50.010705 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7fe12b-ac95-44a2-ad78-59b9fb854c32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a37be821dec241f29ddba0a7b68f2801, user_id=fc7d9d0572e246eea84c626440f66706, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:50.014359 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7fe12b-ac95-44a2-ad78-59b9fb854c32 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:50.050849 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 842/1679] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:46:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:46:50.053196 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e46a53-81a8-48c9-8b53-7f11825e5581 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:50.053704 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e46a53-81a8-48c9-8b53-7f11825e5581 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:50.054009 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 838/1680] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:46:49 2022] PUT /identity/v3/projects/a37be821dec241f29ddba0a7b68f2801/users/fc7d9d0572e246eea84c626440f66706/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:46:50.054342 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e46a53-81a8-48c9-8b53-7f11825e5581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:50.057743 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82993c5a-fb9c-47ce-bbbd-0db4b27b01d7 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:50.061332 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82993c5a-fb9c-47ce-bbbd-0db4b27b01d7 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:50.061332 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82993c5a-fb9c-47ce-bbbd-0db4b27b01d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:50.116259 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-30e46a53-81a8-48c9-8b53-7f11825e5581 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:46:50.118576 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-30e46a53-81a8-48c9-8b53-7f11825e5581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:50.193126 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-82993c5a-fb9c-47ce-bbbd-0db4b27b01d7 None None] MFA Rules not processed for user `f6eaba3a278e4bc99aca0dedd649fa9f`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:46:50.194624 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-82993c5a-fb9c-47ce-bbbd-0db4b27b01d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:50.249400 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 843/1681] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:46:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:46:50.262386 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d92a75-6dad-4ebb-ae5a-4806a6eeff3b None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:50.262667 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d92a75-6dad-4ebb-ae5a-4806a6eeff3b None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:50.262934 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d92a75-6dad-4ebb-ae5a-4806a6eeff3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:50.274405 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:46:50.304507 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-a1d92a75-6dad-4ebb-ae5a-4806a6eeff3b None None] MFA Rules not processed for user `fc7d9d0572e246eea84c626440f66706`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:46:50.306952 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-a1d92a75-6dad-4ebb-ae5a-4806a6eeff3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:50.369563 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:46:50.396988 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 839/1682] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:46:50 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:46:50.401357 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82993c5a-fb9c-47ce-bbbd-0db4b27b01d7 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:50.407197 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-82993c5a-fb9c-47ce-bbbd-0db4b27b01d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:50.442507 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 844/1683] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:46:50 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:46:50.445258 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494dfbc2-5f97-4833-ac7e-1cb64e73237b None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:50.445569 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494dfbc2-5f97-4833-ac7e-1cb64e73237b None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:50.445851 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494dfbc2-5f97-4833-ac7e-1cb64e73237b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:50.511347 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-494dfbc2-5f97-4833-ac7e-1cb64e73237b None None] MFA Rules not processed for user `f6eaba3a278e4bc99aca0dedd649fa9f`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:46:50.513118 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-494dfbc2-5f97-4833-ac7e-1cb64e73237b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:50.529582 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 845/1684] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:46:50 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:46:50.530955 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494dfbc2-5f97-4833-ac7e-1cb64e73237b None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:50.558755 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494dfbc2-5f97-4833-ac7e-1cb64e73237b None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:50.564519 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a6e3a-c345-4960-b7d2-65891f9698a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:50.565337 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a6e3a-c345-4960-b7d2-65891f9698a3 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:50.565616 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82993c5a-fb9c-47ce-bbbd-0db4b27b01d7 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:50.565909 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a6e3a-c345-4960-b7d2-65891f9698a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:50.566346 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a6e3a-c345-4960-b7d2-65891f9698a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:50.567314 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1a6e3a-c345-4960-b7d2-65891f9698a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:50.570060 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1a6e3a-c345-4960-b7d2-65891f9698a3 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:50.572327 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8e112e-0bcf-4421-af09-7571dc6f5f7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:50.573316 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e112e-0bcf-4421-af09-7571dc6f5f7e service neutron] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:50.573810 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e112e-0bcf-4421-af09-7571dc6f5f7e service neutron] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:50.574296 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e112e-0bcf-4421-af09-7571dc6f5f7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:50.584867 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8e112e-0bcf-4421-af09-7571dc6f5f7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:50.586541 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8e112e-0bcf-4421-af09-7571dc6f5f7e service neutron] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:50.602279 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 840/1685] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:46:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:46:50.608442 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8e112e-0bcf-4421-af09-7571dc6f5f7e service neutron] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:50.610032 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-ba8e112e-0bcf-4421-af09-7571dc6f5f7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:50.636153 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 846/1686] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:46:50 2022] POST /identity/v3/projects => generated 377 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:46:50.639172 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a6e3a-c345-4960-b7d2-65891f9698a3 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:50.785952 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8e112e-0bcf-4421-af09-7571dc6f5f7e service neutron] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:50.792694 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a6e3a-c345-4960-b7d2-65891f9698a3 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:50.806752 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df8eb84-989e-4dad-8d8d-3ea6c3d92c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:50.807784 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df8eb84-989e-4dad-8d8d-3ea6c3d92c00 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:50.808214 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df8eb84-989e-4dad-8d8d-3ea6c3d92c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:50.808770 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df8eb84-989e-4dad-8d8d-3ea6c3d92c00 admin admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:50.809558 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df8eb84-989e-4dad-8d8d-3ea6c3d92c00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:50.813673 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df8eb84-989e-4dad-8d8d-3ea6c3d92c00 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:50.860616 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 847/1687] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:46:50 2022] POST /identity/v3/users => generated 295 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:46:50.865019 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df8eb84-989e-4dad-8d8d-3ea6c3d92c00 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:50.877364 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df8eb84-989e-4dad-8d8d-3ea6c3d92c00 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:50.884435 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad479f49-4391-45e5-be87-3b55b9347a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:50.885593 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad479f49-4391-45e5-be87-3b55b9347a7a admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:50.886354 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad479f49-4391-45e5-be87-3b55b9347a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:50.887038 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad479f49-4391-45e5-be87-3b55b9347a7a admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:50.887876 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad479f49-4391-45e5-be87-3b55b9347a7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:50.889899 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad479f49-4391-45e5-be87-3b55b9347a7a admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:50.905039 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 848/1688] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:46:50 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 10:46:50.910257 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad479f49-4391-45e5-be87-3b55b9347a7a admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:50.925846 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad479f49-4391-45e5-be87-3b55b9347a7a admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:50.928064 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:46:50.935546 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a27de70-4b34-4137-bf8f-edf863b95527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:50.938020 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a27de70-4b34-4137-bf8f-edf863b95527 admin admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:50.939634 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a27de70-4b34-4137-bf8f-edf863b95527 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:50.940119 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a27de70-4b34-4137-bf8f-edf863b95527 admin admin] PATH_INFO: `/v3/projects/b97cf9e0fa6a4dd4a4f8222041030624/users/a8b09c040e3c41a6806564565cc994ef/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:50.949376 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e766bbc-b27a-4a6c-859a-75c1f5ff1113 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bd5356829b6454993bab9bad2f79fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:50.951587 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e766bbc-b27a-4a6c-859a-75c1f5ff1113 service nova] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:50.952349 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e766bbc-b27a-4a6c-859a-75c1f5ff1113 service nova] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:50.953411 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e766bbc-b27a-4a6c-859a-75c1f5ff1113 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:50.980376 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a27de70-4b34-4137-bf8f-edf863b95527 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b97cf9e0fa6a4dd4a4f8222041030624, user_id=a8b09c040e3c41a6806564565cc994ef, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:50.985407 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a27de70-4b34-4137-bf8f-edf863b95527 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:51.019943 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 849/1689] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:46:50 2022] PUT /identity/v3/projects/b97cf9e0fa6a4dd4a4f8222041030624/users/a8b09c040e3c41a6806564565cc994ef/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:46:51.021423 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a27de70-4b34-4137-bf8f-edf863b95527 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:51.104194 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e766bbc-b27a-4a6c-859a-75c1f5ff1113 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:51.110934 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e766bbc-b27a-4a6c-859a-75c1f5ff1113 service nova] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:51.134598 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 841/1690] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:46:50 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:46:51.164890 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a27de70-4b34-4137-bf8f-edf863b95527 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:51.175860 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e037552-25a4-4db6-9b86-cc52270d42ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:51.177502 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e037552-25a4-4db6-9b86-cc52270d42ff admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:51.178068 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e037552-25a4-4db6-9b86-cc52270d42ff admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:51.178642 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e037552-25a4-4db6-9b86-cc52270d42ff admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:51.179834 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e037552-25a4-4db6-9b86-cc52270d42ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:51.182042 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e037552-25a4-4db6-9b86-cc52270d42ff admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:51.184672 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:46:51.207082 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e766bbc-b27a-4a6c-859a-75c1f5ff1113 service nova] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:51.209203 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-7e766bbc-b27a-4a6c-859a-75c1f5ff1113 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:51.226675 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 850/1691] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:46:51 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 10:46:51.230666 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e037552-25a4-4db6-9b86-cc52270d42ff admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:51.231171 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e766bbc-b27a-4a6c-859a-75c1f5ff1113 service nova] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:51.238574 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd121399-7313-45c9-8b0e-f833a60eff4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:51.240774 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd121399-7313-45c9-8b0e-f833a60eff4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:51.241303 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd121399-7313-45c9-8b0e-f833a60eff4e admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:51.241973 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd121399-7313-45c9-8b0e-f833a60eff4e admin admin] PATH_INFO: `/v3/projects/7745546a1c6d481a9d0bf4984a536507` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:51.243570 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd121399-7313-45c9-8b0e-f833a60eff4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7745546a1c6d481a9d0bf4984a536507)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:51.250817 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e037552-25a4-4db6-9b86-cc52270d42ff admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:51.268283 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd999355-7752-4e7b-b2d9-f1d3412d1033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:51.269504 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd121399-7313-45c9-8b0e-f833a60eff4e admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:51.269688 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd999355-7752-4e7b-b2d9-f1d3412d1033 admin admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:51.270084 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd999355-7752-4e7b-b2d9-f1d3412d1033 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:51.270366 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd999355-7752-4e7b-b2d9-f1d3412d1033 admin admin] PATH_INFO: `/v3/projects/b97cf9e0fa6a4dd4a4f8222041030624/users/a8b09c040e3c41a6806564565cc994ef/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:51.275293 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd999355-7752-4e7b-b2d9-f1d3412d1033 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b97cf9e0fa6a4dd4a4f8222041030624, user_id=a8b09c040e3c41a6806564565cc994ef, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:51.285600 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd999355-7752-4e7b-b2d9-f1d3412d1033 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:51.325370 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 851/1692] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:46:51 2022] PUT /identity/v3/projects/b97cf9e0fa6a4dd4a4f8222041030624/users/a8b09c040e3c41a6806564565cc994ef/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:46:51.325370 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4b16b5-a92d-4bda-9365-8c1c84512196 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:51.325538 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4b16b5-a92d-4bda-9365-8c1c84512196 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:51.325819 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4b16b5-a92d-4bda-9365-8c1c84512196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:51.408296 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-9b4b16b5-a92d-4bda-9365-8c1c84512196 None None] MFA Rules not processed for user `a8b09c040e3c41a6806564565cc994ef`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:46:51.409958 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-9b4b16b5-a92d-4bda-9365-8c1c84512196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:51.436134 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-fd121399-7313-45c9-8b0e-f833a60eff4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7745546a1c6d481a9d0bf4984a536507', 'request_id': 'req-fd121399-7313-45c9-8b0e-f833a60eff4e'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:51.467113 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-fd121399-7313-45c9-8b0e-f833a60eff4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7745546a1c6d481a9d0bf4984a536507', 'request_id': 'req-fd121399-7313-45c9-8b0e-f833a60eff4e'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:51.498935 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-fd121399-7313-45c9-8b0e-f833a60eff4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7745546a1c6d481a9d0bf4984a536507', 'request_id': 'req-fd121399-7313-45c9-8b0e-f833a60eff4e'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:51.535388 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-fd121399-7313-45c9-8b0e-f833a60eff4e admin admin] The token cache is being invalidate because project 7745546a1c6d481a9d0bf4984a536507 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90653) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:46:51.535717 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-fd121399-7313-45c9-8b0e-f833a60eff4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:51.544103 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-fd121399-7313-45c9-8b0e-f833a60eff4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:51.552117 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 842/1693] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:46:51 2022] DELETE /identity/v3/projects/7745546a1c6d481a9d0bf4984a536507 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 10:46:51.553117 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd121399-7313-45c9-8b0e-f833a60eff4e admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:51.561841 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-fd121399-7313-45c9-8b0e-f833a60eff4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:51.564686 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 852/1694] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:46:51 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 10:46:51.578694 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4b16b5-a92d-4bda-9365-8c1c84512196 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:51.582691 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-9b4b16b5-a92d-4bda-9365-8c1c84512196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:51.731206 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4b16b5-a92d-4bda-9365-8c1c84512196 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:51.739131 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dd83a1-385c-42ed-92f9-8791c28c3b6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:51.740096 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd83a1-385c-42ed-92f9-8791c28c3b6d service cinder] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:51.740528 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd83a1-385c-42ed-92f9-8791c28c3b6d service cinder] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:51.740953 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd83a1-385c-42ed-92f9-8791c28c3b6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:51.743377 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-b2dd83a1-385c-42ed-92f9-8791c28c3b6d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:51.763100 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd121399-7313-45c9-8b0e-f833a60eff4e admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:51.769622 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dd83a1-385c-42ed-92f9-8791c28c3b6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:51.771808 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dd83a1-385c-42ed-92f9-8791c28c3b6d service cinder] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:51.779010 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a94ab5-f7d7-4c10-a03f-e5797abb1108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:51.780194 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a94ab5-f7d7-4c10-a03f-e5797abb1108 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:51.780487 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a94ab5-f7d7-4c10-a03f-e5797abb1108 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:51.780924 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a94ab5-f7d7-4c10-a03f-e5797abb1108 admin admin] PATH_INFO: `/v3/users/9ab443b09fe54923ac8cc15609bdedfc` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:51.784952 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a94ab5-f7d7-4c10-a03f-e5797abb1108 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ab443b09fe54923ac8cc15609bdedfc)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:51.792100 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a94ab5-f7d7-4c10-a03f-e5797abb1108 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:51.796751 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 853/1695] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:46:51 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:46:51.840404 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:46:51.875418 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-44a94ab5-f7d7-4c10-a03f-e5797abb1108 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ab443b09fe54923ac8cc15609bdedfc'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:51.892752 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-44a94ab5-f7d7-4c10-a03f-e5797abb1108 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ab443b09fe54923ac8cc15609bdedfc'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:51.894023 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-44a94ab5-f7d7-4c10-a03f-e5797abb1108 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ab443b09fe54923ac8cc15609bdedfc'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:51.907495 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411485fc-94ed-46df-8de5-a5dd86dbf7be None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:51.907960 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411485fc-94ed-46df-8de5-a5dd86dbf7be None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:51.908034 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411485fc-94ed-46df-8de5-a5dd86dbf7be None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:51.910214 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 854/1696] 10.222.0.15 () {54 vars in 912 bytes} [Sun Sep 11 10:46:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 10:46:51.913285 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-44a94ab5-f7d7-4c10-a03f-e5797abb1108 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ab443b09fe54923ac8cc15609bdedfc'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:51.918724 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c505d-2340-4cb4-8f90-23137e80ba54 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:51.918724 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c505d-2340-4cb4-8f90-23137e80ba54 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:51.919004 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c505d-2340-4cb4-8f90-23137e80ba54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:51.931051 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-44a94ab5-f7d7-4c10-a03f-e5797abb1108 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ab443b09fe54923ac8cc15609bdedfc'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:51.977532 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 843/1697] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:46:51 2022] DELETE /identity/v3/users/9ab443b09fe54923ac8cc15609bdedfc => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:46:51.978469 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a94ab5-f7d7-4c10-a03f-e5797abb1108 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:51.988073 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-44a94ab5-f7d7-4c10-a03f-e5797abb1108 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:51.994377 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-537c505d-2340-4cb4-8f90-23137e80ba54 None None] MFA Rules not processed for user `40072bc8dcb5454382a93e20b63f8594`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:46:51.996282 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-537c505d-2340-4cb4-8f90-23137e80ba54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:52.119495 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 855/1698] 10.222.0.15 () {58 vars in 1009 bytes} [Sun Sep 11 10:46:51 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:46:52.122478 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a94ab5-f7d7-4c10-a03f-e5797abb1108 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:52.132788 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ff831c-0ee9-4735-8413-a12dcee1a550 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:52.134480 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff831c-0ee9-4735-8413-a12dcee1a550 service cinder] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:52.135150 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff831c-0ee9-4735-8413-a12dcee1a550 service cinder] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:52.136449 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff831c-0ee9-4735-8413-a12dcee1a550 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:52.140123 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-d3ff831c-0ee9-4735-8413-a12dcee1a550 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:52.150012 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537c505d-2340-4cb4-8f90-23137e80ba54 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:52.151631 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-537c505d-2340-4cb4-8f90-23137e80ba54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:52.174113 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537c505d-2340-4cb4-8f90-23137e80ba54 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:52.185495 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db27ebe3-6fa0-4ff6-b5e3-56080c5f9450 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:52.186140 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db27ebe3-6fa0-4ff6-b5e3-56080c5f9450 service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:52.186427 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db27ebe3-6fa0-4ff6-b5e3-56080c5f9450 service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:52.186707 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db27ebe3-6fa0-4ff6-b5e3-56080c5f9450 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:52.196612 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:46:52.211022 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db27ebe3-6fa0-4ff6-b5e3-56080c5f9450 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:52.212628 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db27ebe3-6fa0-4ff6-b5e3-56080c5f9450 service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:52.235005 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 856/1699] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:46:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:46:52.252124 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ff831c-0ee9-4735-8413-a12dcee1a550 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:52.256934 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ff831c-0ee9-4735-8413-a12dcee1a550 service cinder] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:52.281699 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 844/1700] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:46:51 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:46:52.710676 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2117958e-e344-4dec-a9f9-3f7ec2e21971 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:52.710967 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2117958e-e344-4dec-a9f9-3f7ec2e21971 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:52.711248 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2117958e-e344-4dec-a9f9-3f7ec2e21971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:52.813410 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-2117958e-e344-4dec-a9f9-3f7ec2e21971 None None] MFA Rules not processed for user `ea9f58a33d1449099bd39f8d09f9f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:46:52.823267 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-2117958e-e344-4dec-a9f9-3f7ec2e21971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:52.998486 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ff831c-0ee9-4735-8413-a12dcee1a550 service cinder] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:53.001447 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-d3ff831c-0ee9-4735-8413-a12dcee1a550 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:53.038890 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 857/1701] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:46:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:46:53.039598 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb643d-5789-4da3-8690-3a3adefa5244 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:53.039598 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb643d-5789-4da3-8690-3a3adefa5244 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:53.041349 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb643d-5789-4da3-8690-3a3adefa5244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:53.116736 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-3bbb643d-5789-4da3-8690-3a3adefa5244 None None] MFA Rules not processed for user `ea9f58a33d1449099bd39f8d09f9f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:46:53.126202 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-3bbb643d-5789-4da3-8690-3a3adefa5244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:53.147758 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:46:53.157277 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 858/1702] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:46:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 10:46:53.157483 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ff831c-0ee9-4735-8413-a12dcee1a550 service cinder] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:53.167902 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbb643d-5789-4da3-8690-3a3adefa5244 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:53.169315 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-3bbb643d-5789-4da3-8690-3a3adefa5244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:53.169603 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff3372c-6b3e-4a62-adc7-b214029b101a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:53.173843 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff3372c-6b3e-4a62-adc7-b214029b101a admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:53.173843 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff3372c-6b3e-4a62-adc7-b214029b101a admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:53.174099 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff3372c-6b3e-4a62-adc7-b214029b101a admin admin] PATH_INFO: `/v3/projects/cfc31fd786a54975ae40a9f4c1c2c8e4` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:53.176759 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff3372c-6b3e-4a62-adc7-b214029b101a admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfc31fd786a54975ae40a9f4c1c2c8e4)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:53.181847 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff3372c-6b3e-4a62-adc7-b214029b101a admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:53.349397 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-6ff3372c-6b3e-4a62-adc7-b214029b101a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfc31fd786a54975ae40a9f4c1c2c8e4', 'request_id': 'req-6ff3372c-6b3e-4a62-adc7-b214029b101a'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:53.371368 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-6ff3372c-6b3e-4a62-adc7-b214029b101a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfc31fd786a54975ae40a9f4c1c2c8e4', 'request_id': 'req-6ff3372c-6b3e-4a62-adc7-b214029b101a'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:53.410080 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-6ff3372c-6b3e-4a62-adc7-b214029b101a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfc31fd786a54975ae40a9f4c1c2c8e4', 'request_id': 'req-6ff3372c-6b3e-4a62-adc7-b214029b101a'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:53.427264 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbb643d-5789-4da3-8690-3a3adefa5244 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:53.454330 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-6ff3372c-6b3e-4a62-adc7-b214029b101a admin admin] The token cache is being invalidate because project cfc31fd786a54975ae40a9f4c1c2c8e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90653) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:46:53.455079 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-6ff3372c-6b3e-4a62-adc7-b214029b101a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:53.456557 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-6ff3372c-6b3e-4a62-adc7-b214029b101a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:53.464186 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 845/1703] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:46:52 2022] DELETE /identity/v3/projects/cfc31fd786a54975ae40a9f4c1c2c8e4 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:46:53.466045 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff3372c-6b3e-4a62-adc7-b214029b101a admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:53.470816 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-6ff3372c-6b3e-4a62-adc7-b214029b101a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:53.620373 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d490b9b-5e52-49c1-90d4-cad0e9e76885 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bd5356829b6454993bab9bad2f79fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:53.621027 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d490b9b-5e52-49c1-90d4-cad0e9e76885 service nova] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:53.621311 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d490b9b-5e52-49c1-90d4-cad0e9e76885 service nova] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:53.621586 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d490b9b-5e52-49c1-90d4-cad0e9e76885 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:53.628619 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-7d490b9b-5e52-49c1-90d4-cad0e9e76885 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:53.657454 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:46:53.666942 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff3372c-6b3e-4a62-adc7-b214029b101a admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:53.681699 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63c6fe7-f725-42fe-a65c-5f33aed7fc5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:53.685956 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63c6fe7-f725-42fe-a65c-5f33aed7fc5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:53.685956 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63c6fe7-f725-42fe-a65c-5f33aed7fc5e admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:53.685956 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63c6fe7-f725-42fe-a65c-5f33aed7fc5e admin admin] PATH_INFO: `/v3/projects/146ff83c68434a60a1c6b655f5d9adfa` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:53.685956 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63c6fe7-f725-42fe-a65c-5f33aed7fc5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=146ff83c68434a60a1c6b655f5d9adfa)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:53.716781 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63c6fe7-f725-42fe-a65c-5f33aed7fc5e admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:53.887330 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d490b9b-5e52-49c1-90d4-cad0e9e76885 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:53.889014 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d490b9b-5e52-49c1-90d4-cad0e9e76885 service nova] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:53.919365 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 859/1704] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:46:53 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 752 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 10:46:53.921225 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4066799-56b2-480a-8507-6f04d12a601b None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:53.921673 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4066799-56b2-480a-8507-6f04d12a601b None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:53.922231 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4066799-56b2-480a-8507-6f04d12a601b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:53.956629 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-c4066799-56b2-480a-8507-6f04d12a601b None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:46:53.959905 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-c4066799-56b2-480a-8507-6f04d12a601b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:53.976775 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-b63c6fe7-f725-42fe-a65c-5f33aed7fc5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '146ff83c68434a60a1c6b655f5d9adfa', 'request_id': 'req-b63c6fe7-f725-42fe-a65c-5f33aed7fc5e'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:53.990769 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-b63c6fe7-f725-42fe-a65c-5f33aed7fc5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '146ff83c68434a60a1c6b655f5d9adfa', 'request_id': 'req-b63c6fe7-f725-42fe-a65c-5f33aed7fc5e'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:54.009935 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-b63c6fe7-f725-42fe-a65c-5f33aed7fc5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '146ff83c68434a60a1c6b655f5d9adfa', 'request_id': 'req-b63c6fe7-f725-42fe-a65c-5f33aed7fc5e'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:54.044061 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-b63c6fe7-f725-42fe-a65c-5f33aed7fc5e admin admin] The token cache is being invalidate because project 146ff83c68434a60a1c6b655f5d9adfa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90653) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:46:54.044369 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-b63c6fe7-f725-42fe-a65c-5f33aed7fc5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:54.048953 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-b63c6fe7-f725-42fe-a65c-5f33aed7fc5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:54.054115 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 846/1705] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:46:53 2022] DELETE /identity/v3/projects/146ff83c68434a60a1c6b655f5d9adfa => generated 0 bytes in 588 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:46:54.058830 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62212c50-acaf-4834-9bc2-428c94768f59 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:54.059143 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62212c50-acaf-4834-9bc2-428c94768f59 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:54.059441 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62212c50-acaf-4834-9bc2-428c94768f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:54.103090 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 860/1706] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:46:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:46:54.105334 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4066799-56b2-480a-8507-6f04d12a601b None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:54.110931 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-c4066799-56b2-480a-8507-6f04d12a601b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:54.134367 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-62212c50-acaf-4834-9bc2-428c94768f59 None None] MFA Rules not processed for user `8d0af2b5f354439d92bfa317a0baefd5`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:46:54.135654 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-62212c50-acaf-4834-9bc2-428c94768f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:54.179208 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:46:54.293258 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 847/1707] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:46:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:46:54.298095 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb79ce2-956a-4176-bef6-2cc9775409e2 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:54.298356 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb79ce2-956a-4176-bef6-2cc9775409e2 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:54.298601 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb79ce2-956a-4176-bef6-2cc9775409e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:54.316812 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4066799-56b2-480a-8507-6f04d12a601b None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:54.343118 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4ca241-ae5c-40f6-b647-968936cb1091 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08477b4c88f04c9e97391aba401879e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:54.344802 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4ca241-ae5c-40f6-b647-968936cb1091 service placement] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:54.347522 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4ca241-ae5c-40f6-b647-968936cb1091 service placement] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:54.347705 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4ca241-ae5c-40f6-b647-968936cb1091 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:54.356313 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-4cb79ce2-956a-4176-bef6-2cc9775409e2 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:46:54.358422 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-7a4ca241-ae5c-40f6-b647-968936cb1091 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:54.359197 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-4cb79ce2-956a-4176-bef6-2cc9775409e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:54.379207 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 848/1708] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:46:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:46:54.384901 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8b0201-e70d-441f-8c51-edb9bf77c69c None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:54.385188 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8b0201-e70d-441f-8c51-edb9bf77c69c None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:54.385773 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8b0201-e70d-441f-8c51-edb9bf77c69c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:54.414333 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4ca241-ae5c-40f6-b647-968936cb1091 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:54.416002 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4ca241-ae5c-40f6-b647-968936cb1091 service placement] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:54.436526 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-2c8b0201-e70d-441f-8c51-edb9bf77c69c None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:46:54.438839 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-2c8b0201-e70d-441f-8c51-edb9bf77c69c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:54.451660 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 861/1709] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:46:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:46:54.452514 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4ca241-ae5c-40f6-b647-968936cb1091 service placement] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:54.464557 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-7a4ca241-ae5c-40f6-b647-968936cb1091 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:54.508492 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 849/1710] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:46:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:46:54.515639 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c48cefe-6b8b-412a-9e32-50428c3d0169 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:54.516253 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c48cefe-6b8b-412a-9e32-50428c3d0169 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:54.516742 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c48cefe-6b8b-412a-9e32-50428c3d0169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:54.524080 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4ca241-ae5c-40f6-b647-968936cb1091 service placement] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:54.553199 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7242907-a071-4413-88ef-a56053e1a858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:54.553885 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7242907-a071-4413-88ef-a56053e1a858 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:54.554244 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7242907-a071-4413-88ef-a56053e1a858 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:54.554523 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7242907-a071-4413-88ef-a56053e1a858 admin admin] PATH_INFO: `/v3/users/fe55415017ae433cb76b49b164a13ef6` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:54.565104 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7242907-a071-4413-88ef-a56053e1a858 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe55415017ae433cb76b49b164a13ef6)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:54.567485 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7242907-a071-4413-88ef-a56053e1a858 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:54.595787 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-6c48cefe-6b8b-412a-9e32-50428c3d0169 None None] MFA Rules not processed for user `f6eaba3a278e4bc99aca0dedd649fa9f`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:46:54.600888 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-6c48cefe-6b8b-412a-9e32-50428c3d0169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:54.746725 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:46:54.792136 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-c7242907-a071-4413-88ef-a56053e1a858 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe55415017ae433cb76b49b164a13ef6'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:54.816098 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-c7242907-a071-4413-88ef-a56053e1a858 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe55415017ae433cb76b49b164a13ef6'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:54.818076 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-c7242907-a071-4413-88ef-a56053e1a858 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe55415017ae433cb76b49b164a13ef6'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:54.819754 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-c7242907-a071-4413-88ef-a56053e1a858 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe55415017ae433cb76b49b164a13ef6'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:54.877468 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 850/1711] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:46:54 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 367 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:46:54.879942 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670c9817-2de3-4220-848a-e56b008301a3 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:54.881501 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670c9817-2de3-4220-848a-e56b008301a3 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:54.886571 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670c9817-2de3-4220-848a-e56b008301a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:54.892418 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-c7242907-a071-4413-88ef-a56053e1a858 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe55415017ae433cb76b49b164a13ef6'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:54.920900 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 862/1712] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:46:54 2022] DELETE /identity/v3/users/fe55415017ae433cb76b49b164a13ef6 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:46:54.924251 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7242907-a071-4413-88ef-a56053e1a858 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:54.929382 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-c7242907-a071-4413-88ef-a56053e1a858 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:54.935839 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-670c9817-2de3-4220-848a-e56b008301a3 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:46:54.938196 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-670c9817-2de3-4220-848a-e56b008301a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:55.045925 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7242907-a071-4413-88ef-a56053e1a858 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:55.060306 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cd5b65-d5df-4ed5-8561-602948791066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:55.060953 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd5b65-d5df-4ed5-8561-602948791066 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:55.062042 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd5b65-d5df-4ed5-8561-602948791066 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:55.062327 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd5b65-d5df-4ed5-8561-602948791066 admin admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:55.065129 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cd5b65-d5df-4ed5-8561-602948791066 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:46:55.065510 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cd5b65-d5df-4ed5-8561-602948791066 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:55.069942 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cd5b65-d5df-4ed5-8561-602948791066 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:55.071815 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 851/1713] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:46:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:46:55.073268 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553dada5-c90d-4681-9e96-b88c12a73887 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:55.073555 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553dada5-c90d-4681-9e96-b88c12a73887 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:55.073829 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553dada5-c90d-4681-9e96-b88c12a73887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:55.099386 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 863/1714] 10.222.0.15 () {58 vars in 1153 bytes} [Sun Sep 11 10:46:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:46:55.104664 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cd5b65-d5df-4ed5-8561-602948791066 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:55.117912 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-553dada5-c90d-4681-9e96-b88c12a73887 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:46:55.118139 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cd5b65-d5df-4ed5-8561-602948791066 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:55.122372 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-553dada5-c90d-4681-9e96-b88c12a73887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:55.123989 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d502e721-dc83-4867-989f-167fed91232d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:55.124641 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d502e721-dc83-4867-989f-167fed91232d admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:55.124930 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d502e721-dc83-4867-989f-167fed91232d admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:55.125436 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d502e721-dc83-4867-989f-167fed91232d admin admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:55.126273 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d502e721-dc83-4867-989f-167fed91232d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:46:55.127404 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d502e721-dc83-4867-989f-167fed91232d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:55.129080 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d502e721-dc83-4867-989f-167fed91232d admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:55.144550 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 852/1715] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:46:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:46:55.146478 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 864/1716] 10.222.0.15 () {58 vars in 1153 bytes} [Sun Sep 11 10:46:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:46:55.149152 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553dada5-c90d-4681-9e96-b88c12a73887 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:55.153254 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d502e721-dc83-4867-989f-167fed91232d admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:55.165807 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d502e721-dc83-4867-989f-167fed91232d admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:55.171207 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f9f871-c279-40ac-8343-2a82734941ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:55.172496 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f9f871-c279-40ac-8343-2a82734941ec admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:55.172496 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f9f871-c279-40ac-8343-2a82734941ec admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:55.172723 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f9f871-c279-40ac-8343-2a82734941ec admin admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:55.173315 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f9f871-c279-40ac-8343-2a82734941ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:55.175898 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553dada5-c90d-4681-9e96-b88c12a73887 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:55.176571 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f9f871-c279-40ac-8343-2a82734941ec admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:55.188246 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6692773-9733-4460-b5f5-c596f738f234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:55.189993 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6692773-9733-4460-b5f5-c596f738f234 admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:55.190397 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6692773-9733-4460-b5f5-c596f738f234 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:55.191087 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6692773-9733-4460-b5f5-c596f738f234 admin admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:55.195069 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6692773-9733-4460-b5f5-c596f738f234 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:55.196833 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6692773-9733-4460-b5f5-c596f738f234 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:55.229189 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 865/1717] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:46:55 2022] POST /identity/v3/projects => generated 367 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:46:55.230485 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9deaf4-505f-49a5-8f2b-0fabfd8d094a None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:55.230786 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9deaf4-505f-49a5-8f2b-0fabfd8d094a None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:55.231065 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9deaf4-505f-49a5-8f2b-0fabfd8d094a None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:55.235740 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 866/1718] 10.222.0.15 () {54 vars in 912 bytes} [Sun Sep 11 10:46:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:46:55.236789 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9deaf4-505f-49a5-8f2b-0fabfd8d094a None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:55.244894 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-be9deaf4-505f-49a5-8f2b-0fabfd8d094a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:55.254108 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 853/1719] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:46:55 2022] POST /identity/v3/projects => generated 381 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:46:55.254108 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6692773-9733-4460-b5f5-c596f738f234 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:55.317325 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:46:55.367467 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6692773-9733-4460-b5f5-c596f738f234 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:55.374498 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9deaf4-505f-49a5-8f2b-0fabfd8d094a None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:55.374790 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44faba7-81c2-47c1-9bfc-14fb694b3ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:55.376059 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44faba7-81c2-47c1-9bfc-14fb694b3ba4 admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:55.376540 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44faba7-81c2-47c1-9bfc-14fb694b3ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:55.379479 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452075fa-e931-4e48-aa86-4ac2cada14fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:55.379773 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44faba7-81c2-47c1-9bfc-14fb694b3ba4 admin admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:55.380508 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452075fa-e931-4e48-aa86-4ac2cada14fb service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:55.380746 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44faba7-81c2-47c1-9bfc-14fb694b3ba4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:55.381161 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452075fa-e931-4e48-aa86-4ac2cada14fb service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:55.381620 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452075fa-e931-4e48-aa86-4ac2cada14fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:55.385014 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44faba7-81c2-47c1-9bfc-14fb694b3ba4 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:55.395354 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452075fa-e931-4e48-aa86-4ac2cada14fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:55.399700 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452075fa-e931-4e48-aa86-4ac2cada14fb service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:55.427881 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 867/1720] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:46:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:46:55.431627 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae15621-a282-46a7-b9f5-fe15d68cae67 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:55.433610 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae15621-a282-46a7-b9f5-fe15d68cae67 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:55.434427 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae15621-a282-46a7-b9f5-fe15d68cae67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:55.443526 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 854/1721] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:46:55 2022] POST /identity/v3/users => generated 284 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:46:55.446218 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44faba7-81c2-47c1-9bfc-14fb694b3ba4 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:55.476605 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44faba7-81c2-47c1-9bfc-14fb694b3ba4 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:55.478404 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-0ae15621-a282-46a7-b9f5-fe15d68cae67 None None] MFA Rules not processed for user `40072bc8dcb5454382a93e20b63f8594`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:46:55.480942 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-0ae15621-a282-46a7-b9f5-fe15d68cae67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:55.491950 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38242ff4-9b04-4d04-82f0-7665add41dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:55.493329 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38242ff4-9b04-4d04-82f0-7665add41dff admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:55.493867 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38242ff4-9b04-4d04-82f0-7665add41dff admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:55.494389 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38242ff4-9b04-4d04-82f0-7665add41dff admin admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:55.495660 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38242ff4-9b04-4d04-82f0-7665add41dff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:55.498075 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38242ff4-9b04-4d04-82f0-7665add41dff admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:55.522367 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 868/1722] 10.222.0.15 () {58 vars in 1009 bytes} [Sun Sep 11 10:46:55 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:46:55.526430 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f44d3-7ca5-469c-8407-d7b64ec18996 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:55.527417 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f44d3-7ca5-469c-8407-d7b64ec18996 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:55.530078 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f44d3-7ca5-469c-8407-d7b64ec18996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:55.567205 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 855/1723] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:46:55 2022] POST /identity/v3/users => generated 291 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:46:55.570247 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38242ff4-9b04-4d04-82f0-7665add41dff admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:55.610757 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38242ff4-9b04-4d04-82f0-7665add41dff admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:55.633310 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90d3c55-ef06-4f88-a411-c7e284a89a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:55.635543 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90d3c55-ef06-4f88-a411-c7e284a89a4a admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:55.636015 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90d3c55-ef06-4f88-a411-c7e284a89a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:55.636452 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90d3c55-ef06-4f88-a411-c7e284a89a4a admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:55.637361 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90d3c55-ef06-4f88-a411-c7e284a89a4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:55.639551 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90d3c55-ef06-4f88-a411-c7e284a89a4a admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:55.649789 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-230f44d3-7ca5-469c-8407-d7b64ec18996 None None] MFA Rules not processed for user `2238a62cbf1846999b31ea9434b67743`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:46:55.653577 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-230f44d3-7ca5-469c-8407-d7b64ec18996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:55.670439 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 856/1724] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:46:55 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:46:55.671445 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90d3c55-ef06-4f88-a411-c7e284a89a4a admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:55.675132 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:46:55.693923 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90d3c55-ef06-4f88-a411-c7e284a89a4a admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:55.700550 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d222089-38cd-49e5-8cac-34e18cece0c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:55.700550 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d222089-38cd-49e5-8cac-34e18cece0c6 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:55.700931 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d222089-38cd-49e5-8cac-34e18cece0c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:55.701203 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d222089-38cd-49e5-8cac-34e18cece0c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:55.701857 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d222089-38cd-49e5-8cac-34e18cece0c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:55.703857 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d222089-38cd-49e5-8cac-34e18cece0c6 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:55.719582 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 857/1725] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:46:55 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:46:55.724315 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d222089-38cd-49e5-8cac-34e18cece0c6 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:55.730396 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-2d222089-38cd-49e5-8cac-34e18cece0c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:55.776063 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d222089-38cd-49e5-8cac-34e18cece0c6 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:55.781681 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8198181e-3635-45f7-933d-221553a40af3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:55.782528 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8198181e-3635-45f7-933d-221553a40af3 service neutron] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:55.782986 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8198181e-3635-45f7-933d-221553a40af3 service neutron] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:55.783445 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8198181e-3635-45f7-933d-221553a40af3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:55.795181 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8198181e-3635-45f7-933d-221553a40af3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:55.800750 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8198181e-3635-45f7-933d-221553a40af3 service neutron] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:55.828618 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 858/1726] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:46:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:46:55.830666 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8198181e-3635-45f7-933d-221553a40af3 service neutron] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:55.847913 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8198181e-3635-45f7-933d-221553a40af3 service neutron] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:55.851930 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 869/1727] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:46:55 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:46:55.851930 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230f44d3-7ca5-469c-8407-d7b64ec18996 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:55.860188 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eae4c24-3e60-43be-9322-f3a4f441e061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:55.863581 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eae4c24-3e60-43be-9322-f3a4f441e061 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:55.864106 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eae4c24-3e60-43be-9322-f3a4f441e061 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:55.866768 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eae4c24-3e60-43be-9322-f3a4f441e061 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:55.868017 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eae4c24-3e60-43be-9322-f3a4f441e061 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:55.871556 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230f44d3-7ca5-469c-8407-d7b64ec18996 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:55.873059 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eae4c24-3e60-43be-9322-f3a4f441e061 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:55.889183 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 859/1728] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:46:55 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:46:55.899678 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eae4c24-3e60-43be-9322-f3a4f441e061 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:55.903386 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a964af18-03b7-4c87-8484-4c9918797f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:55.904192 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a964af18-03b7-4c87-8484-4c9918797f82 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:55.911386 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a964af18-03b7-4c87-8484-4c9918797f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:55.912395 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a964af18-03b7-4c87-8484-4c9918797f82 admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:55.914105 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a964af18-03b7-4c87-8484-4c9918797f82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:55.915884 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a964af18-03b7-4c87-8484-4c9918797f82 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:55.916694 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eae4c24-3e60-43be-9322-f3a4f441e061 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:55.922805 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3545f82c-d154-480d-b543-0ed576af5d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:55.923722 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545f82c-d154-480d-b543-0ed576af5d72 admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:55.923904 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545f82c-d154-480d-b543-0ed576af5d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:55.924214 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545f82c-d154-480d-b543-0ed576af5d72 admin admin] PATH_INFO: `/v3/projects/c086fe2e5e8246e5b0077d8e283f3891/users/8bb4827fc2d54c568cd499892bf09b89/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:55.939880 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 870/1729] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:46:55 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:46:55.944548 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a964af18-03b7-4c87-8484-4c9918797f82 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:55.982576 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3545f82c-d154-480d-b543-0ed576af5d72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c086fe2e5e8246e5b0077d8e283f3891, user_id=8bb4827fc2d54c568cd499892bf09b89, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:55.984185 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a964af18-03b7-4c87-8484-4c9918797f82 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:55.985455 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3545f82c-d154-480d-b543-0ed576af5d72 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:56.002673 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e0b9fb-8fa9-4f77-9f81-c4b3d7a099d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:56.004951 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e0b9fb-8fa9-4f77-9f81-c4b3d7a099d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:56.004951 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e0b9fb-8fa9-4f77-9f81-c4b3d7a099d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:56.005165 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e0b9fb-8fa9-4f77-9f81-c4b3d7a099d0 admin admin] PATH_INFO: `/v3/projects/495be81242fd48ca97f6353d69d54dca/users/c26deb12229c4df3bebc3477c0f622a9/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:56.035905 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 860/1730] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:46:55 2022] PUT /identity/v3/projects/c086fe2e5e8246e5b0077d8e283f3891/users/8bb4827fc2d54c568cd499892bf09b89/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:46:56.038261 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7f1963-1b1f-4a65-9726-27de12ece784 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:56.038627 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7f1963-1b1f-4a65-9726-27de12ece784 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:56.038911 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7f1963-1b1f-4a65-9726-27de12ece784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:56.072621 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e0b9fb-8fa9-4f77-9f81-c4b3d7a099d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=495be81242fd48ca97f6353d69d54dca, user_id=c26deb12229c4df3bebc3477c0f622a9, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:56.084407 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-ae7f1963-1b1f-4a65-9726-27de12ece784 None None] MFA Rules not processed for user `8bb4827fc2d54c568cd499892bf09b89`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:46:56.086183 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-ae7f1963-1b1f-4a65-9726-27de12ece784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:56.088029 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e0b9fb-8fa9-4f77-9f81-c4b3d7a099d0 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:56.118500 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 871/1731] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:46:55 2022] PUT /identity/v3/projects/495be81242fd48ca97f6353d69d54dca/users/c26deb12229c4df3bebc3477c0f622a9/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:46:56.135998 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5a82f6-673c-4f7b-b727-7f833f51c667 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:56.137668 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5a82f6-673c-4f7b-b727-7f833f51c667 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:56.138140 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5a82f6-673c-4f7b-b727-7f833f51c667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:56.251089 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 861/1732] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:46:56 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:46:56.258837 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7f1963-1b1f-4a65-9726-27de12ece784 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:56.259661 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-ec5a82f6-673c-4f7b-b727-7f833f51c667 None None] MFA Rules not processed for user `c26deb12229c4df3bebc3477c0f622a9`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:46:56.260961 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-ec5a82f6-673c-4f7b-b727-7f833f51c667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:56.303882 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:46:56.430088 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7f1963-1b1f-4a65-9726-27de12ece784 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:56.505141 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 872/1733] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:46:56 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 372 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:46:56.515521 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5a82f6-673c-4f7b-b727-7f833f51c667 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:56.568678 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d0e888-b4c2-4433-a390-63eb57286a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:56.569324 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d0e888-b4c2-4433-a390-63eb57286a92 admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:56.569608 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d0e888-b4c2-4433-a390-63eb57286a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:56.569918 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d0e888-b4c2-4433-a390-63eb57286a92 admin admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:56.570694 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d0e888-b4c2-4433-a390-63eb57286a92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:56.575243 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d0e888-b4c2-4433-a390-63eb57286a92 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:56.642206 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 862/1734] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:46:56 2022] POST /identity/v3/projects => generated 367 bytes in 384 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:46:56.645266 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d0e888-b4c2-4433-a390-63eb57286a92 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:56.647398 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-f7d0e888-b4c2-4433-a390-63eb57286a92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:56.751107 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5a82f6-673c-4f7b-b727-7f833f51c667 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:56.772774 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:46:56.840995 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d0e888-b4c2-4433-a390-63eb57286a92 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:56.849184 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460f68a2-6bdc-489d-bea6-187199c60b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:56.850094 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460f68a2-6bdc-489d-bea6-187199c60b80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:56.850554 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460f68a2-6bdc-489d-bea6-187199c60b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:56.850935 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460f68a2-6bdc-489d-bea6-187199c60b80 admin admin] PATH_INFO: `/v3/projects/d8af004e81e544519c5d040babb51fe1` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:56.852268 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460f68a2-6bdc-489d-bea6-187199c60b80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8af004e81e544519c5d040babb51fe1)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:56.855182 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460f68a2-6bdc-489d-bea6-187199c60b80 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:56.885396 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abadd89-841f-4ede-8064-47ffbaa6b209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:56.886147 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abadd89-841f-4ede-8064-47ffbaa6b209 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:56.886506 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abadd89-841f-4ede-8064-47ffbaa6b209 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:56.886786 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abadd89-841f-4ede-8064-47ffbaa6b209 admin admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:56.887589 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abadd89-841f-4ede-8064-47ffbaa6b209 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:56.892996 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abadd89-841f-4ede-8064-47ffbaa6b209 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:56.950123 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-460f68a2-6bdc-489d-bea6-187199c60b80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8af004e81e544519c5d040babb51fe1', 'request_id': 'req-460f68a2-6bdc-489d-bea6-187199c60b80'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:56.952281 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:46:56.957769 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 873/1735] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:46:56 2022] POST /identity/v3/projects => generated 381 bytes in 447 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 10:46:56.958787 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abadd89-841f-4ede-8064-47ffbaa6b209 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:56.976929 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-460f68a2-6bdc-489d-bea6-187199c60b80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8af004e81e544519c5d040babb51fe1', 'request_id': 'req-460f68a2-6bdc-489d-bea6-187199c60b80'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:56.999488 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-460f68a2-6bdc-489d-bea6-187199c60b80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8af004e81e544519c5d040babb51fe1', 'request_id': 'req-460f68a2-6bdc-489d-bea6-187199c60b80'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:57.026794 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-460f68a2-6bdc-489d-bea6-187199c60b80 admin admin] The token cache is being invalidate because project d8af004e81e544519c5d040babb51fe1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90653) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:46:57.027087 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-460f68a2-6bdc-489d-bea6-187199c60b80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:57.036183 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-460f68a2-6bdc-489d-bea6-187199c60b80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:46:57.040192 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 863/1736] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:46:56 2022] DELETE /identity/v3/projects/d8af004e81e544519c5d040babb51fe1 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:46:57.046818 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460f68a2-6bdc-489d-bea6-187199c60b80 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:57.054964 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-460f68a2-6bdc-489d-bea6-187199c60b80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:57.161469 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abadd89-841f-4ede-8064-47ffbaa6b209 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:57.180047 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ee8936-ab6f-4f96-a755-0ac7608d935b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:57.181850 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ee8936-ab6f-4f96-a755-0ac7608d935b admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:57.182158 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ee8936-ab6f-4f96-a755-0ac7608d935b admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:57.183041 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ee8936-ab6f-4f96-a755-0ac7608d935b admin admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:57.185632 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ee8936-ab6f-4f96-a755-0ac7608d935b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:57.206467 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ee8936-ab6f-4f96-a755-0ac7608d935b admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:57.303159 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 874/1737] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:46:56 2022] POST /identity/v3/users => generated 290 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:46:57.305820 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5314ea-aed0-4bda-af2e-05aaeb90678a None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:57.306312 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5314ea-aed0-4bda-af2e-05aaeb90678a None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:57.307915 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5314ea-aed0-4bda-af2e-05aaeb90678a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:57.367395 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-ac5314ea-aed0-4bda-af2e-05aaeb90678a None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:46:57.371117 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-ac5314ea-aed0-4bda-af2e-05aaeb90678a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:57.371884 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460f68a2-6bdc-489d-bea6-187199c60b80 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:57.395177 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 875/1738] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:46:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:46:57.399550 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5314ea-aed0-4bda-af2e-05aaeb90678a None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:57.401358 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-ac5314ea-aed0-4bda-af2e-05aaeb90678a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:57.402148 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fe2114-9492-4fbc-b2eb-3a2f5b90c95a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:57.402811 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fe2114-9492-4fbc-b2eb-3a2f5b90c95a admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:57.403093 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fe2114-9492-4fbc-b2eb-3a2f5b90c95a admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:57.403386 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fe2114-9492-4fbc-b2eb-3a2f5b90c95a admin admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:57.410700 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fe2114-9492-4fbc-b2eb-3a2f5b90c95a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:57.412728 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fe2114-9492-4fbc-b2eb-3a2f5b90c95a admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:57.457065 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5314ea-aed0-4bda-af2e-05aaeb90678a None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:57.484125 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 864/1739] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:46:57 2022] POST /identity/v3/users => generated 297 bytes in 443 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:46:57.485507 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a30bc-c2fd-4ee6-b138-2578686f113a None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:57.485801 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a30bc-c2fd-4ee6-b138-2578686f113a None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:57.486896 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a30bc-c2fd-4ee6-b138-2578686f113a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:57.489856 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b8c834-5012-4697-8326-da14fc05176f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:57.490960 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b8c834-5012-4697-8326-da14fc05176f admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:57.491240 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b8c834-5012-4697-8326-da14fc05176f admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:57.491998 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b8c834-5012-4697-8326-da14fc05176f admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:57.492231 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b8c834-5012-4697-8326-da14fc05176f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:57.494953 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b8c834-5012-4697-8326-da14fc05176f admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:57.549478 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-431a30bc-c2fd-4ee6-b138-2578686f113a None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:46:57.550801 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-431a30bc-c2fd-4ee6-b138-2578686f113a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:57.554080 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 876/1740] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:46:57 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:46:57.555110 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b8c834-5012-4697-8326-da14fc05176f admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:57.572486 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 865/1741] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:46:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:46:57.573593 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431a30bc-c2fd-4ee6-b138-2578686f113a None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:57.573965 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b8c834-5012-4697-8326-da14fc05176f admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:57.594723 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a2e65b-c047-4979-b2e8-62fdb606f258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:57.595710 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a2e65b-c047-4979-b2e8-62fdb606f258 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:57.596565 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a2e65b-c047-4979-b2e8-62fdb606f258 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:57.597209 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a2e65b-c047-4979-b2e8-62fdb606f258 admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:57.598394 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a2e65b-c047-4979-b2e8-62fdb606f258 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:57.600333 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a2e65b-c047-4979-b2e8-62fdb606f258 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:57.605171 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431a30bc-c2fd-4ee6-b138-2578686f113a None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:57.619140 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 877/1742] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:46:57 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:46:57.620173 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a2e65b-c047-4979-b2e8-62fdb606f258 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:57.624858 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5cdf41-65e4-4474-b859-00f7be8dec05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:57.625561 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5cdf41-65e4-4474-b859-00f7be8dec05 admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:57.625846 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5cdf41-65e4-4474-b859-00f7be8dec05 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:57.626615 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5cdf41-65e4-4474-b859-00f7be8dec05 admin admin] PATH_INFO: `/v3/projects/b3bd5059c53941c0bc4deba0fa92cce1/users/435029a94a684f7a8ddb439996bc5220/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:57.639018 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a2e65b-c047-4979-b2e8-62fdb606f258 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:57.666273 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376aa40d-d511-4f9a-bc66-c4feee93e0e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:57.667066 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376aa40d-d511-4f9a-bc66-c4feee93e0e6 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:57.667357 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376aa40d-d511-4f9a-bc66-c4feee93e0e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:57.667662 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376aa40d-d511-4f9a-bc66-c4feee93e0e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:57.668329 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376aa40d-d511-4f9a-bc66-c4feee93e0e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:46:57.668607 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5cdf41-65e4-4474-b859-00f7be8dec05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3bd5059c53941c0bc4deba0fa92cce1, user_id=435029a94a684f7a8ddb439996bc5220, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:57.668720 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376aa40d-d511-4f9a-bc66-c4feee93e0e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:57.670241 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376aa40d-d511-4f9a-bc66-c4feee93e0e6 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:57.675513 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5cdf41-65e4-4474-b859-00f7be8dec05 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:57.714245 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 866/1743] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:46:57 2022] PUT /identity/v3/projects/b3bd5059c53941c0bc4deba0fa92cce1/users/435029a94a684f7a8ddb439996bc5220/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:46:57.715476 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5cdf41-65e4-4474-b859-00f7be8dec05 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:57.715982 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 878/1744] 10.222.0.15 () {58 vars in 1153 bytes} [Sun Sep 11 10:46:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:46:57.717031 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376aa40d-d511-4f9a-bc66-c4feee93e0e6 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:57.746529 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376aa40d-d511-4f9a-bc66-c4feee93e0e6 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:57.852698 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:46:57.872323 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5cdf41-65e4-4474-b859-00f7be8dec05 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:57.880102 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39bd083-8508-46a4-9359-71e0d5530209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:57.880870 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39bd083-8508-46a4-9359-71e0d5530209 admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:57.881164 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39bd083-8508-46a4-9359-71e0d5530209 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:57.881506 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39bd083-8508-46a4-9359-71e0d5530209 admin admin] PATH_INFO: `/v3/projects/7ae4b30a9e304c92ae0185cfea78f4e3/users/23b8587a62174d6aa6951a8d64c1edb9/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:57.883057 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc89f3e4-2313-4407-af8f-87cce22aaa5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:57.884007 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc89f3e4-2313-4407-af8f-87cce22aaa5b admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:57.884771 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc89f3e4-2313-4407-af8f-87cce22aaa5b admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:57.885295 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc89f3e4-2313-4407-af8f-87cce22aaa5b admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:57.886172 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc89f3e4-2313-4407-af8f-87cce22aaa5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:57.888340 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc89f3e4-2313-4407-af8f-87cce22aaa5b admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:57.902541 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 879/1745] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:46:57 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:46:57.904467 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc89f3e4-2313-4407-af8f-87cce22aaa5b admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:57.923069 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc89f3e4-2313-4407-af8f-87cce22aaa5b admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:57.928551 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39bd083-8508-46a4-9359-71e0d5530209 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ae4b30a9e304c92ae0185cfea78f4e3, user_id=23b8587a62174d6aa6951a8d64c1edb9, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:57.931396 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39bd083-8508-46a4-9359-71e0d5530209 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:57.932407 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfc8b77-9cf3-4a43-8e47-c1c5a68dd85a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:57.933343 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfc8b77-9cf3-4a43-8e47-c1c5a68dd85a admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:57.934160 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfc8b77-9cf3-4a43-8e47-c1c5a68dd85a admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:57.934654 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfc8b77-9cf3-4a43-8e47-c1c5a68dd85a admin admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:57.936116 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfc8b77-9cf3-4a43-8e47-c1c5a68dd85a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:57.941565 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfc8b77-9cf3-4a43-8e47-c1c5a68dd85a admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:57.956894 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 867/1746] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:46:57 2022] PUT /identity/v3/projects/7ae4b30a9e304c92ae0185cfea78f4e3/users/23b8587a62174d6aa6951a8d64c1edb9/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:46:57.959290 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39bd083-8508-46a4-9359-71e0d5530209 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:58.010508 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 880/1747] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:46:57 2022] POST /identity/v3/projects => generated 373 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:46:58.011471 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfc8b77-9cf3-4a43-8e47-c1c5a68dd85a admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:58.023804 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=91,keystone:INSERT=6,keystone:DELETE=3 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:46:58.146986 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39bd083-8508-46a4-9359-71e0d5530209 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:58.205062 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfc8b77-9cf3-4a43-8e47-c1c5a68dd85a admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:58.212945 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef148280-48ab-4bf9-8dd6-e2924090059f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:58.213580 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef148280-48ab-4bf9-8dd6-e2924090059f admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:58.214025 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef148280-48ab-4bf9-8dd6-e2924090059f admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:58.214477 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef148280-48ab-4bf9-8dd6-e2924090059f admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:58.215138 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef148280-48ab-4bf9-8dd6-e2924090059f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:58.217096 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef148280-48ab-4bf9-8dd6-e2924090059f admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:58.241463 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 881/1748] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:46:58 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:46:58.242366 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef148280-48ab-4bf9-8dd6-e2924090059f admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:58.246736 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9a1414-1314-40fd-83c4-08cbaa1118a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:58.247657 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9a1414-1314-40fd-83c4-08cbaa1118a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:58.248189 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9a1414-1314-40fd-83c4-08cbaa1118a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:58.248519 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9a1414-1314-40fd-83c4-08cbaa1118a3 admin admin] PATH_INFO: `/v3/projects/b3bd5059c53941c0bc4deba0fa92cce1/users/435029a94a684f7a8ddb439996bc5220/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:58.253662 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9a1414-1314-40fd-83c4-08cbaa1118a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3bd5059c53941c0bc4deba0fa92cce1, user_id=435029a94a684f7a8ddb439996bc5220, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:58.256729 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9a1414-1314-40fd-83c4-08cbaa1118a3 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:58.272644 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef148280-48ab-4bf9-8dd6-e2924090059f admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:58.294287 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 868/1749] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:46:57 2022] PUT /identity/v3/projects/b3bd5059c53941c0bc4deba0fa92cce1/users/435029a94a684f7a8ddb439996bc5220/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:46:58.298103 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9a1414-1314-40fd-83c4-08cbaa1118a3 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:58.446506 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3021c6f-87f0-4747-b427-4a1ac2fdba56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:58.447149 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3021c6f-87f0-4747-b427-4a1ac2fdba56 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:58.447454 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3021c6f-87f0-4747-b427-4a1ac2fdba56 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:58.447804 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3021c6f-87f0-4747-b427-4a1ac2fdba56 admin admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:58.448465 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3021c6f-87f0-4747-b427-4a1ac2fdba56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:58.456113 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9a1414-1314-40fd-83c4-08cbaa1118a3 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:58.458752 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3021c6f-87f0-4747-b427-4a1ac2fdba56 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:58.466878 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea176e2b-64a2-4310-8aed-d0250477f86a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:58.468776 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea176e2b-64a2-4310-8aed-d0250477f86a admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:58.469310 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea176e2b-64a2-4310-8aed-d0250477f86a admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:58.469821 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea176e2b-64a2-4310-8aed-d0250477f86a admin admin] PATH_INFO: `/v3/projects/7ae4b30a9e304c92ae0185cfea78f4e3/users/23b8587a62174d6aa6951a8d64c1edb9/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:58.473292 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea176e2b-64a2-4310-8aed-d0250477f86a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ae4b30a9e304c92ae0185cfea78f4e3, user_id=23b8587a62174d6aa6951a8d64c1edb9, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:58.476731 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea176e2b-64a2-4310-8aed-d0250477f86a admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:58.492217 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 869/1750] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:46:58 2022] PUT /identity/v3/projects/7ae4b30a9e304c92ae0185cfea78f4e3/users/23b8587a62174d6aa6951a8d64c1edb9/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:46:58.494065 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee81df-b7dc-48f1-ae55-6c064bc47d1a None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:58.494729 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee81df-b7dc-48f1-ae55-6c064bc47d1a None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:58.495110 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee81df-b7dc-48f1-ae55-6c064bc47d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:58.516954 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 882/1751] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:46:58 2022] POST /identity/v3/users => generated 287 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:46:58.518639 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a22f96-131f-400f-86ad-058f6627cfcd None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:58.520750 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a22f96-131f-400f-86ad-058f6627cfcd None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:58.521368 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a22f96-131f-400f-86ad-058f6627cfcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:58.574433 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-feee81df-b7dc-48f1-ae55-6c064bc47d1a None None] MFA Rules not processed for user `435029a94a684f7a8ddb439996bc5220`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:46:58.577227 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-feee81df-b7dc-48f1-ae55-6c064bc47d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:58.589850 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-70a22f96-131f-400f-86ad-058f6627cfcd None None] MFA Rules not processed for user `23b8587a62174d6aa6951a8d64c1edb9`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:46:58.591450 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-70a22f96-131f-400f-86ad-058f6627cfcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:58.708676 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:46:58.719574 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:46:58.799337 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 883/1752] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:46:58 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:46:58.802280 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 870/1753] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:46:58 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:46:58.810127 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a22f96-131f-400f-86ad-058f6627cfcd None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:58.819977 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feee81df-b7dc-48f1-ae55-6c064bc47d1a None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:58.828389 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-feee81df-b7dc-48f1-ae55-6c064bc47d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:58.958643 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feee81df-b7dc-48f1-ae55-6c064bc47d1a None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:58.964382 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d1ab58-f37f-4660-a538-7c286db5e843 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:58.965029 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d1ab58-f37f-4660-a538-7c286db5e843 service cinder] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:58.965313 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d1ab58-f37f-4660-a538-7c286db5e843 service cinder] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:58.965590 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d1ab58-f37f-4660-a538-7c286db5e843 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:58.980701 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d1ab58-f37f-4660-a538-7c286db5e843 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:58.983289 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d1ab58-f37f-4660-a538-7c286db5e843 service cinder] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:59.012753 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 871/1754] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:46:58 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:46:59.016931 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d1ab58-f37f-4660-a538-7c286db5e843 service cinder] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:59.017249 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a22f96-131f-400f-86ad-058f6627cfcd None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:59.026371 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-64d1ab58-f37f-4660-a538-7c286db5e843 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:59.032022 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c9d155-1e62-4a11-9641-5ce720184584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:59.033061 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c9d155-1e62-4a11-9641-5ce720184584 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:59.033499 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c9d155-1e62-4a11-9641-5ce720184584 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:59.036295 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c9d155-1e62-4a11-9641-5ce720184584 admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:59.037146 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c9d155-1e62-4a11-9641-5ce720184584 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:59.049242 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c9d155-1e62-4a11-9641-5ce720184584 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:59.071658 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d1ab58-f37f-4660-a538-7c286db5e843 service cinder] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:59.093061 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fae429-ee6a-45c4-ab46-a4e2d6f98493 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:59.093564 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fae429-ee6a-45c4-ab46-a4e2d6f98493 service cinder] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:59.095521 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 884/1755] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:46:58 2022] GET /identity/v3/roles => generated 1420 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:46:59.095849 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fae429-ee6a-45c4-ab46-a4e2d6f98493 service cinder] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:59.096707 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c9d155-1e62-4a11-9641-5ce720184584 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:59.099850 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fae429-ee6a-45c4-ab46-a4e2d6f98493 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:59.117023 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c9d155-1e62-4a11-9641-5ce720184584 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:59.138495 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03e920e-ae82-4b6b-bb02-93d7d352a97b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:59.139295 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03e920e-ae82-4b6b-bb02-93d7d352a97b admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:59.139761 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03e920e-ae82-4b6b-bb02-93d7d352a97b admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:59.140226 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03e920e-ae82-4b6b-bb02-93d7d352a97b admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:59.141033 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03e920e-ae82-4b6b-bb02-93d7d352a97b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:59.142784 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03e920e-ae82-4b6b-bb02-93d7d352a97b admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:59.147933 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fae429-ee6a-45c4-ab46-a4e2d6f98493 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:59.154889 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fae429-ee6a-45c4-ab46-a4e2d6f98493 service cinder] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:59.180321 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 885/1756] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:46:59 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:46:59.188399 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03e920e-ae82-4b6b-bb02-93d7d352a97b admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:59.203315 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 872/1757] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:46:59 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:46:59.238762 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fae429-ee6a-45c4-ab46-a4e2d6f98493 service cinder] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:59.239741 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03e920e-ae82-4b6b-bb02-93d7d352a97b admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:59.251113 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d3bfcc-15dd-4117-bdd0-460418fc052b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:59.251869 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d3bfcc-15dd-4117-bdd0-460418fc052b admin admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:59.252146 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d3bfcc-15dd-4117-bdd0-460418fc052b admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:59.252424 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d3bfcc-15dd-4117-bdd0-460418fc052b admin admin] PATH_INFO: `/v3/projects/4faa31fc51a348a4accee557406bbc5a/users/e0f337fe25d74a069a1c65029264f5fe/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:59.257322 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fae429-ee6a-45c4-ab46-a4e2d6f98493 service cinder] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:59.268061 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3b3034-4860-4ac6-a177-05123a622965 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:46:59.269947 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3b3034-4860-4ac6-a177-05123a622965 service cinder] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:59.271195 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3b3034-4860-4ac6-a177-05123a622965 service cinder] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:59.271635 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3b3034-4860-4ac6-a177-05123a622965 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:59.290400 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-cc3b3034-4860-4ac6-a177-05123a622965 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:59.298197 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d3bfcc-15dd-4117-bdd0-460418fc052b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4faa31fc51a348a4accee557406bbc5a, user_id=e0f337fe25d74a069a1c65029264f5fe, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:59.317062 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d3bfcc-15dd-4117-bdd0-460418fc052b admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:59.439324 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 886/1758] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:46:59 2022] PUT /identity/v3/projects/4faa31fc51a348a4accee557406bbc5a/users/e0f337fe25d74a069a1c65029264f5fe/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:46:59.451214 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d5ee0d-95f2-412b-b830-c94a9f10baf6 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:46:59.451552 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d5ee0d-95f2-412b-b830-c94a9f10baf6 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:46:59.451825 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d5ee0d-95f2-412b-b830-c94a9f10baf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:46:59.512936 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3b3034-4860-4ac6-a177-05123a622965 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:46:59.512936 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3b3034-4860-4ac6-a177-05123a622965 service cinder] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:46:59.515957 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-44d5ee0d-95f2-412b-b830-c94a9f10baf6 None None] MFA Rules not processed for user `e0f337fe25d74a069a1c65029264f5fe`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:46:59.516736 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-44d5ee0d-95f2-412b-b830-c94a9f10baf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:59.534854 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 873/1759] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:46:59 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:46:59.647009 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 887/1760] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:46:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 10:46:59.660874 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3b3034-4860-4ac6-a177-05123a622965 service cinder] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:46:59.663775 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-cc3b3034-4860-4ac6-a177-05123a622965 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:46:59.898981 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3b3034-4860-4ac6-a177-05123a622965 service cinder] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:46:59.907768 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:00.130950 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f52397-8149-47b7-8f13-797495154a76 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bd5356829b6454993bab9bad2f79fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:00.131818 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f52397-8149-47b7-8f13-797495154a76 service nova] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:00.132137 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f52397-8149-47b7-8f13-797495154a76 service nova] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:00.132663 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f52397-8149-47b7-8f13-797495154a76 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:00.153947 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f52397-8149-47b7-8f13-797495154a76 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:00.163215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f52397-8149-47b7-8f13-797495154a76 service nova] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:00.213573 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 874/1761] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:46:59 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 554 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:47:00.669750 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d5ee0d-95f2-412b-b830-c94a9f10baf6 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:00.738797 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:00.834596 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d5ee0d-95f2-412b-b830-c94a9f10baf6 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:00.854399 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337d869c-9b73-42e7-a693-7de6c66ee8ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:00.863648 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d869c-9b73-42e7-a693-7de6c66ee8ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:00.868662 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d869c-9b73-42e7-a693-7de6c66ee8ae admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:00.868941 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d869c-9b73-42e7-a693-7de6c66ee8ae admin admin] PATH_INFO: `/v3/users/e0f337fe25d74a069a1c65029264f5fe` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:00.890973 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337d869c-9b73-42e7-a693-7de6c66ee8ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0f337fe25d74a069a1c65029264f5fe)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:00.893054 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337d869c-9b73-42e7-a693-7de6c66ee8ae admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:01.083725 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-337d869c-9b73-42e7-a693-7de6c66ee8ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0f337fe25d74a069a1c65029264f5fe'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:01.112851 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-337d869c-9b73-42e7-a693-7de6c66ee8ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0f337fe25d74a069a1c65029264f5fe'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:01.114349 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-337d869c-9b73-42e7-a693-7de6c66ee8ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0f337fe25d74a069a1c65029264f5fe'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:01.115178 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-337d869c-9b73-42e7-a693-7de6c66ee8ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0f337fe25d74a069a1c65029264f5fe'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:01.157183 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-337d869c-9b73-42e7-a693-7de6c66ee8ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0f337fe25d74a069a1c65029264f5fe'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:01.181971 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 888/1762] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:47:00 2022] DELETE /identity/v3/users/e0f337fe25d74a069a1c65029264f5fe => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:47:01.185010 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1860f6c1-e4a6-4da4-9b0e-10812aef19c5 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:01.185010 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1860f6c1-e4a6-4da4-9b0e-10812aef19c5 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:01.185010 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1860f6c1-e4a6-4da4-9b0e-10812aef19c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:01.226438 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-1860f6c1-e4a6-4da4-9b0e-10812aef19c5 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:47:01.229352 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-1860f6c1-e4a6-4da4-9b0e-10812aef19c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:01.561959 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 875/1763] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:47:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:47:01.569519 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337d869c-9b73-42e7-a693-7de6c66ee8ae admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:01.570982 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-337d869c-9b73-42e7-a693-7de6c66ee8ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:01.740909 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337d869c-9b73-42e7-a693-7de6c66ee8ae admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:01.749544 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784fbf9d-c5d6-4e3c-bead-f67ecff93525 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:01.750540 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784fbf9d-c5d6-4e3c-bead-f67ecff93525 service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:01.752090 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784fbf9d-c5d6-4e3c-bead-f67ecff93525 service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:01.753155 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784fbf9d-c5d6-4e3c-bead-f67ecff93525 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:01.776599 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784fbf9d-c5d6-4e3c-bead-f67ecff93525 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:01.778329 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784fbf9d-c5d6-4e3c-bead-f67ecff93525 service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:01.806107 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 889/1764] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:47:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:47:02.232538 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ca4ee-1416-4def-a07d-e36dad034c69 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:02.233500 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ca4ee-1416-4def-a07d-e36dad034c69 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:02.234112 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ca4ee-1416-4def-a07d-e36dad034c69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:02.494582 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-0e1ca4ee-1416-4def-a07d-e36dad034c69 None None] MFA Rules not processed for user `bb44e7aef0c7436d93a7233307b5f095`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:47:02.503472 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-0e1ca4ee-1416-4def-a07d-e36dad034c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:02.731296 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 876/1765] 10.222.0.15 () {58 vars in 1009 bytes} [Sun Sep 11 10:47:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 500 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:47:03.029799 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784fbf9d-c5d6-4e3c-bead-f67ecff93525 service neutron] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:03.031543 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-784fbf9d-c5d6-4e3c-bead-f67ecff93525 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:03.138576 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1ca4ee-1416-4def-a07d-e36dad034c69 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:03.140084 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-0e1ca4ee-1416-4def-a07d-e36dad034c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:03.182545 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1ca4ee-1416-4def-a07d-e36dad034c69 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:03.195631 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:03.200563 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95e26a2-8113-46cd-9734-8a9fc30806ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:03.201793 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e26a2-8113-46cd-9734-8a9fc30806ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:03.202413 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e26a2-8113-46cd-9734-8a9fc30806ae admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:03.205152 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e26a2-8113-46cd-9734-8a9fc30806ae admin admin] PATH_INFO: `/v3/users/8bb4827fc2d54c568cd499892bf09b89` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:03.209761 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95e26a2-8113-46cd-9734-8a9fc30806ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bb4827fc2d54c568cd499892bf09b89)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:03.212061 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95e26a2-8113-46cd-9734-8a9fc30806ae admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:03.220689 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:03.308134 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784fbf9d-c5d6-4e3c-bead-f67ecff93525 service neutron] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:03.323030 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db37636f-9478-4295-8279-3ca32348d577 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7ca566e66a45b29e3d02f47bf84981', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:03.324310 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-f95e26a2-8113-46cd-9734-8a9fc30806ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bb4827fc2d54c568cd499892bf09b89'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:03.326192 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37636f-9478-4295-8279-3ca32348d577 service swift] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:03.326709 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37636f-9478-4295-8279-3ca32348d577 service swift] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:03.327405 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37636f-9478-4295-8279-3ca32348d577 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:03.342588 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-f95e26a2-8113-46cd-9734-8a9fc30806ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bb4827fc2d54c568cd499892bf09b89'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:03.343658 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-f95e26a2-8113-46cd-9734-8a9fc30806ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bb4827fc2d54c568cd499892bf09b89'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:03.344319 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db37636f-9478-4295-8279-3ca32348d577 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:03.344612 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-f95e26a2-8113-46cd-9734-8a9fc30806ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bb4827fc2d54c568cd499892bf09b89'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:03.346473 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db37636f-9478-4295-8279-3ca32348d577 service swift] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:03.359203 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 890/1766] 10.222.0.15 () {60 vars in 1384 bytes} [Sun Sep 11 10:47:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 10:47:03.360674 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-f95e26a2-8113-46cd-9734-8a9fc30806ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bb4827fc2d54c568cd499892bf09b89'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:03.394251 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db37636f-9478-4295-8279-3ca32348d577 service swift] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:03.405118 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-db37636f-9478-4295-8279-3ca32348d577 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:03.415766 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 877/1767] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:47:03 2022] DELETE /identity/v3/users/8bb4827fc2d54c568cd499892bf09b89 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:47:03.422212 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a37b6e-16a9-40ae-956e-c0100426ef44 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:03.422794 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a37b6e-16a9-40ae-956e-c0100426ef44 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:03.423311 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a37b6e-16a9-40ae-956e-c0100426ef44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:03.448988 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db37636f-9478-4295-8279-3ca32348d577 service swift] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:03.478969 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-b0a37b6e-16a9-40ae-956e-c0100426ef44 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:47:03.480301 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-b0a37b6e-16a9-40ae-956e-c0100426ef44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:03.682492 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6306c2-899e-4319-be7b-e4525ece8de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:03.685107 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6306c2-899e-4319-be7b-e4525ece8de6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:03.685685 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6306c2-899e-4319-be7b-e4525ece8de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:03.686379 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6306c2-899e-4319-be7b-e4525ece8de6 admin admin] PATH_INFO: `/v3/users/c26deb12229c4df3bebc3477c0f622a9` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:03.692477 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6306c2-899e-4319-be7b-e4525ece8de6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c26deb12229c4df3bebc3477c0f622a9)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:03.699392 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6306c2-899e-4319-be7b-e4525ece8de6 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:03.816392 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 878/1768] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:47:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:47:03.883778 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-6f6306c2-899e-4319-be7b-e4525ece8de6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c26deb12229c4df3bebc3477c0f622a9'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:03.912082 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-6f6306c2-899e-4319-be7b-e4525ece8de6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c26deb12229c4df3bebc3477c0f622a9'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:03.913108 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-6f6306c2-899e-4319-be7b-e4525ece8de6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c26deb12229c4df3bebc3477c0f622a9'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:03.913900 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-6f6306c2-899e-4319-be7b-e4525ece8de6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c26deb12229c4df3bebc3477c0f622a9'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:03.984075 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-6f6306c2-899e-4319-be7b-e4525ece8de6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c26deb12229c4df3bebc3477c0f622a9'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:04.022747 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 891/1769] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:47:03 2022] DELETE /identity/v3/users/c26deb12229c4df3bebc3477c0f622a9 => generated 0 bytes in 630 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 10:47:04.027564 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb44b743-e36b-445b-a48a-df5879590187 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:04.027852 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb44b743-e36b-445b-a48a-df5879590187 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:04.028151 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb44b743-e36b-445b-a48a-df5879590187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:04.056787 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6306c2-899e-4319-be7b-e4525ece8de6 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:04.058790 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-6f6306c2-899e-4319-be7b-e4525ece8de6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:04.097449 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-fb44b743-e36b-445b-a48a-df5879590187 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:47:04.098736 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-fb44b743-e36b-445b-a48a-df5879590187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:04.165878 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:04.236736 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6306c2-899e-4319-be7b-e4525ece8de6 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:04.254276 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f647ae3-e0cb-4d10-afdd-f2eba40670b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:04.258048 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f647ae3-e0cb-4d10-afdd-f2eba40670b3 service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:04.258453 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f647ae3-e0cb-4d10-afdd-f2eba40670b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:04.259592 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f647ae3-e0cb-4d10-afdd-f2eba40670b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:04.269101 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-7f647ae3-e0cb-4d10-afdd-f2eba40670b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:04.273593 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 879/1770] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:47:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:47:04.336369 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f647ae3-e0cb-4d10-afdd-f2eba40670b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:04.349175 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f647ae3-e0cb-4d10-afdd-f2eba40670b3 service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:04.438133 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb44b743-e36b-445b-a48a-df5879590187 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:04.453876 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb44b743-e36b-445b-a48a-df5879590187 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:04.456347 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 892/1771] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:47:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 10:47:04.523704 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ca531c-847b-49d9-a5f7-3e461c0c41e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:04.525786 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ca531c-847b-49d9-a5f7-3e461c0c41e9 service neutron] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:04.528101 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ca531c-847b-49d9-a5f7-3e461c0c41e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:04.530298 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ca531c-847b-49d9-a5f7-3e461c0c41e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:04.588052 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ca531c-847b-49d9-a5f7-3e461c0c41e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:04.595833 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ca531c-847b-49d9-a5f7-3e461c0c41e9 service neutron] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:04.653539 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 880/1772] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:47:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 10:47:04.809387 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f647ae3-e0cb-4d10-afdd-f2eba40670b3 service neutron] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:04.812936 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-7f647ae3-e0cb-4d10-afdd-f2eba40670b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:04.813645 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73bec49-9768-4844-9be3-e5a597b66a45 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:04.813943 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73bec49-9768-4844-9be3-e5a597b66a45 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:04.814220 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73bec49-9768-4844-9be3-e5a597b66a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:04.864057 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f647ae3-e0cb-4d10-afdd-f2eba40670b3 service neutron] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:04.874039 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:04.899339 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-e73bec49-9768-4844-9be3-e5a597b66a45 None None] MFA Rules not processed for user `40072bc8dcb5454382a93e20b63f8594`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:47:04.900685 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-e73bec49-9768-4844-9be3-e5a597b66a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:04.946712 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 881/1773] 10.222.0.25 () {58 vars in 1015 bytes} [Sun Sep 11 10:47:04 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 10:47:04.956435 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575d7b3e-4821-4fa8-b19b-7745a6315117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:04.957585 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575d7b3e-4821-4fa8-b19b-7745a6315117 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:04.957959 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575d7b3e-4821-4fa8-b19b-7745a6315117 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:04.961690 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575d7b3e-4821-4fa8-b19b-7745a6315117 admin admin] PATH_INFO: `/v3/projects/4faa31fc51a348a4accee557406bbc5a` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:04.968897 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575d7b3e-4821-4fa8-b19b-7745a6315117 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4faa31fc51a348a4accee557406bbc5a)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:04.974383 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: WARNING py.warnings [None req-575d7b3e-4821-4fa8-b19b-7745a6315117 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 11 10:47:04.974383 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: warnings.warn(msg) Sep 11 10:47:04.974383 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: Sep 11 10:47:04.980197 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575d7b3e-4821-4fa8-b19b-7745a6315117 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:05.219772 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-575d7b3e-4821-4fa8-b19b-7745a6315117 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4faa31fc51a348a4accee557406bbc5a', 'request_id': 'req-575d7b3e-4821-4fa8-b19b-7745a6315117'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:05.245221 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73bec49-9768-4844-9be3-e5a597b66a45 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:05.246784 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-e73bec49-9768-4844-9be3-e5a597b66a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:05.261020 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-575d7b3e-4821-4fa8-b19b-7745a6315117 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4faa31fc51a348a4accee557406bbc5a', 'request_id': 'req-575d7b3e-4821-4fa8-b19b-7745a6315117'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:05.273595 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-575d7b3e-4821-4fa8-b19b-7745a6315117 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4faa31fc51a348a4accee557406bbc5a', 'request_id': 'req-575d7b3e-4821-4fa8-b19b-7745a6315117'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:05.279886 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73bec49-9768-4844-9be3-e5a597b66a45 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:05.349251 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-575d7b3e-4821-4fa8-b19b-7745a6315117 admin admin] The token cache is being invalidate because project 4faa31fc51a348a4accee557406bbc5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:47:05.349752 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-575d7b3e-4821-4fa8-b19b-7745a6315117 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:05.352528 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-575d7b3e-4821-4fa8-b19b-7745a6315117 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:05.359198 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 893/1774] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:47:04 2022] DELETE /identity/v3/projects/4faa31fc51a348a4accee557406bbc5a => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:47:05.364664 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918e3efe-d1e6-4830-93f2-5cd9f7e89a82 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:05.365221 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918e3efe-d1e6-4830-93f2-5cd9f7e89a82 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:05.365515 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918e3efe-d1e6-4830-93f2-5cd9f7e89a82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:05.411368 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-918e3efe-d1e6-4830-93f2-5cd9f7e89a82 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:47:05.414196 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-918e3efe-d1e6-4830-93f2-5cd9f7e89a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:05.467104 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85678a7f-1b7e-497b-8d67-c2e0b0ab892c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:05.467791 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85678a7f-1b7e-497b-8d67-c2e0b0ab892c service neutron] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:05.468183 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85678a7f-1b7e-497b-8d67-c2e0b0ab892c service neutron] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:05.469013 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85678a7f-1b7e-497b-8d67-c2e0b0ab892c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:05.475596 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-85678a7f-1b7e-497b-8d67-c2e0b0ab892c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:05.501644 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85678a7f-1b7e-497b-8d67-c2e0b0ab892c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:05.503595 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85678a7f-1b7e-497b-8d67-c2e0b0ab892c service neutron] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:05.528378 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 882/1775] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:47:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:47:05.592130 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 894/1776] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:47:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:47:05.600991 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab083a8-6ee5-41eb-b600-1912b0dd3c91 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:05.601586 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab083a8-6ee5-41eb-b600-1912b0dd3c91 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:05.602078 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab083a8-6ee5-41eb-b600-1912b0dd3c91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:05.640263 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-9ab083a8-6ee5-41eb-b600-1912b0dd3c91 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:47:05.642857 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-9ab083a8-6ee5-41eb-b600-1912b0dd3c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:05.688266 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 883/1777] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:47:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:47:05.691347 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918e3efe-d1e6-4830-93f2-5cd9f7e89a82 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:05.709203 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918e3efe-d1e6-4830-93f2-5cd9f7e89a82 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:05.719885 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8319d3-de73-48a4-aee6-5f5a999de971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:05.720522 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8319d3-de73-48a4-aee6-5f5a999de971 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:05.720813 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8319d3-de73-48a4-aee6-5f5a999de971 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:05.721108 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8319d3-de73-48a4-aee6-5f5a999de971 admin admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:05.722086 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8319d3-de73-48a4-aee6-5f5a999de971 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:47:05.722596 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8319d3-de73-48a4-aee6-5f5a999de971 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:05.730891 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8319d3-de73-48a4-aee6-5f5a999de971 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:05.786973 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 895/1778] 10.222.0.15 () {58 vars in 1153 bytes} [Sun Sep 11 10:47:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:47:05.791069 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab083a8-6ee5-41eb-b600-1912b0dd3c91 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:05.804358 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab083a8-6ee5-41eb-b600-1912b0dd3c91 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:05.839915 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df23242-704f-476b-b21a-9e39b2f93eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:05.841997 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df23242-704f-476b-b21a-9e39b2f93eae admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:05.843249 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df23242-704f-476b-b21a-9e39b2f93eae admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:05.844477 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df23242-704f-476b-b21a-9e39b2f93eae admin admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:05.845383 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df23242-704f-476b-b21a-9e39b2f93eae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:05.849459 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df23242-704f-476b-b21a-9e39b2f93eae admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:05.896011 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 884/1779] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:47:05 2022] POST /identity/v3/projects => generated 365 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:47:05.900114 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8319d3-de73-48a4-aee6-5f5a999de971 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:05.933175 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8319d3-de73-48a4-aee6-5f5a999de971 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:06.129466 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df23242-704f-476b-b21a-9e39b2f93eae admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:06.131879 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-0df23242-704f-476b-b21a-9e39b2f93eae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:06.136878 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:06.199496 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4332c8af-66c2-4455-a2ce-f3f0d3565b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:06.200455 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4332c8af-66c2-4455-a2ce-f3f0d3565b50 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:06.202913 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4332c8af-66c2-4455-a2ce-f3f0d3565b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:06.203818 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4332c8af-66c2-4455-a2ce-f3f0d3565b50 admin admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:06.204820 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4332c8af-66c2-4455-a2ce-f3f0d3565b50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:06.213640 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4332c8af-66c2-4455-a2ce-f3f0d3565b50 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:06.297501 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 896/1780] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:47:05 2022] POST /identity/v3/users => generated 283 bytes in 396 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 10:47:06.301635 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4332c8af-66c2-4455-a2ce-f3f0d3565b50 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:06.349599 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df23242-704f-476b-b21a-9e39b2f93eae admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:06.367343 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1734827-7a31-456d-84ca-5f8c0a747b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:06.369061 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1734827-7a31-456d-84ca-5f8c0a747b36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:06.369369 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1734827-7a31-456d-84ca-5f8c0a747b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:06.369628 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1734827-7a31-456d-84ca-5f8c0a747b36 admin admin] PATH_INFO: `/v3/users/fc7d9d0572e246eea84c626440f66706` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:06.373198 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4332c8af-66c2-4455-a2ce-f3f0d3565b50 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:06.376223 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1734827-7a31-456d-84ca-5f8c0a747b36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc7d9d0572e246eea84c626440f66706)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:06.383596 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1734827-7a31-456d-84ca-5f8c0a747b36 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:06.392893 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764e55f8-6f40-4b0a-8047-57bb3244c4a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:06.394166 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764e55f8-6f40-4b0a-8047-57bb3244c4a0 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:06.394321 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764e55f8-6f40-4b0a-8047-57bb3244c4a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:06.396291 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764e55f8-6f40-4b0a-8047-57bb3244c4a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:06.397133 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764e55f8-6f40-4b0a-8047-57bb3244c4a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:06.400787 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764e55f8-6f40-4b0a-8047-57bb3244c4a0 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:06.429573 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 897/1781] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:47:06 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:47:06.430992 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764e55f8-6f40-4b0a-8047-57bb3244c4a0 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:06.446461 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764e55f8-6f40-4b0a-8047-57bb3244c4a0 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:06.454279 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71148cee-47ad-4400-b20d-f1c0d8cf6b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:06.455134 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71148cee-47ad-4400-b20d-f1c0d8cf6b07 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:06.456780 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71148cee-47ad-4400-b20d-f1c0d8cf6b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:06.457337 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71148cee-47ad-4400-b20d-f1c0d8cf6b07 admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:06.458227 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71148cee-47ad-4400-b20d-f1c0d8cf6b07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:06.460394 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71148cee-47ad-4400-b20d-f1c0d8cf6b07 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:06.503465 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 898/1782] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:47:06 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:47:06.511907 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71148cee-47ad-4400-b20d-f1c0d8cf6b07 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:06.533921 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-b1734827-7a31-456d-84ca-5f8c0a747b36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc7d9d0572e246eea84c626440f66706'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:06.535534 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71148cee-47ad-4400-b20d-f1c0d8cf6b07 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:06.556225 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-b1734827-7a31-456d-84ca-5f8c0a747b36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc7d9d0572e246eea84c626440f66706'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:06.556535 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b029731c-4e23-491f-b3b7-abe555da83d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:06.557359 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b029731c-4e23-491f-b3b7-abe555da83d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:06.557874 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b029731c-4e23-491f-b3b7-abe555da83d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:06.558244 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b029731c-4e23-491f-b3b7-abe555da83d4 admin admin] PATH_INFO: `/v3/projects/e093fa22c59b4cbd8498f0187067c598/users/2856de2342bc4bc9a9fa91f1cb7aa1fb/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:06.558498 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-b1734827-7a31-456d-84ca-5f8c0a747b36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc7d9d0572e246eea84c626440f66706'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:06.559845 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-b1734827-7a31-456d-84ca-5f8c0a747b36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc7d9d0572e246eea84c626440f66706'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:06.583142 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-b1734827-7a31-456d-84ca-5f8c0a747b36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc7d9d0572e246eea84c626440f66706'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:06.606246 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b029731c-4e23-491f-b3b7-abe555da83d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e093fa22c59b4cbd8498f0187067c598, user_id=2856de2342bc4bc9a9fa91f1cb7aa1fb, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:06.606897 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:06.621090 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b029731c-4e23-491f-b3b7-abe555da83d4 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:06.657368 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 885/1783] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:47:06 2022] DELETE /identity/v3/users/fc7d9d0572e246eea84c626440f66706 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:47:06.660009 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a433c19c-8791-4989-996f-5fbd5b32c760 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:06.664325 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a433c19c-8791-4989-996f-5fbd5b32c760 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:06.664783 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a433c19c-8791-4989-996f-5fbd5b32c760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:06.674502 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 899/1784] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:47:06 2022] PUT /identity/v3/projects/e093fa22c59b4cbd8498f0187067c598/users/2856de2342bc4bc9a9fa91f1cb7aa1fb/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:47:06.694049 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce2bdc4-53b9-4737-ae96-b19d9a2e7cdd None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:06.694473 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce2bdc4-53b9-4737-ae96-b19d9a2e7cdd None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:06.694867 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce2bdc4-53b9-4737-ae96-b19d9a2e7cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:06.723571 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-a433c19c-8791-4989-996f-5fbd5b32c760 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:47:06.724918 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-a433c19c-8791-4989-996f-5fbd5b32c760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:06.820420 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-8ce2bdc4-53b9-4737-ae96-b19d9a2e7cdd None None] MFA Rules not processed for user `2856de2342bc4bc9a9fa91f1cb7aa1fb`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:47:06.822793 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-8ce2bdc4-53b9-4737-ae96-b19d9a2e7cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:06.929366 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 886/1785] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:47:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:47:07.113059 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 900/1786] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:47:06 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 423 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:47:07.114130 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a433c19c-8791-4989-996f-5fbd5b32c760 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:07.116725 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-a433c19c-8791-4989-996f-5fbd5b32c760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:07.139515 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce2bdc4-53b9-4737-ae96-b19d9a2e7cdd None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:07.141012 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-8ce2bdc4-53b9-4737-ae96-b19d9a2e7cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:07.149633 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a433c19c-8791-4989-996f-5fbd5b32c760 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:07.164358 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571da690-b55b-46e9-b5be-76d85653b52e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:07.164885 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571da690-b55b-46e9-b5be-76d85653b52e admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:07.165163 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571da690-b55b-46e9-b5be-76d85653b52e admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:07.165432 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571da690-b55b-46e9-b5be-76d85653b52e admin admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:07.166082 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571da690-b55b-46e9-b5be-76d85653b52e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:07.170217 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571da690-b55b-46e9-b5be-76d85653b52e admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:07.239156 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 887/1787] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:47:07 2022] POST /identity/v3/projects => generated 365 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:47:07.243166 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571da690-b55b-46e9-b5be-76d85653b52e admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:07.317283 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce2bdc4-53b9-4737-ae96-b19d9a2e7cdd None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:07.412694 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:07.456470 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e01246a-4718-4d76-9151-a614e99ed27d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:07.457889 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e01246a-4718-4d76-9151-a614e99ed27d service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:07.458187 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e01246a-4718-4d76-9151-a614e99ed27d service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:07.459304 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e01246a-4718-4d76-9151-a614e99ed27d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:07.487751 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571da690-b55b-46e9-b5be-76d85653b52e admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:07.512145 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0cf41d-2afb-49aa-acb4-621aa1eb2772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:07.512839 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0cf41d-2afb-49aa-acb4-621aa1eb2772 admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:07.513100 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0cf41d-2afb-49aa-acb4-621aa1eb2772 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:07.513378 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0cf41d-2afb-49aa-acb4-621aa1eb2772 admin admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:07.514029 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0cf41d-2afb-49aa-acb4-621aa1eb2772 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:07.516142 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0cf41d-2afb-49aa-acb4-621aa1eb2772 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:07.603437 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 888/1788] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:47:07 2022] POST /identity/v3/users => generated 289 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:47:07.604377 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0cf41d-2afb-49aa-acb4-621aa1eb2772 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:07.630639 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0cf41d-2afb-49aa-acb4-621aa1eb2772 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:07.646946 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fecfc42-2049-4bf9-a236-2ceffcc975b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:07.647607 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fecfc42-2049-4bf9-a236-2ceffcc975b9 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:07.647894 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fecfc42-2049-4bf9-a236-2ceffcc975b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:07.648172 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fecfc42-2049-4bf9-a236-2ceffcc975b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:07.648831 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fecfc42-2049-4bf9-a236-2ceffcc975b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:07.650456 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fecfc42-2049-4bf9-a236-2ceffcc975b9 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:07.684593 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e01246a-4718-4d76-9151-a614e99ed27d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:07.689075 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e01246a-4718-4d76-9151-a614e99ed27d service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:07.718800 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 901/1789] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:47:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 580 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 10:47:07.752676 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 889/1790] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:47:07 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:47:07.758632 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e01246a-4718-4d76-9151-a614e99ed27d service neutron] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:07.775786 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e01246a-4718-4d76-9151-a614e99ed27d service neutron] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:07.783612 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110daa27-1a48-4d1f-a4ad-beb8a68f3fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:07.786278 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110daa27-1a48-4d1f-a4ad-beb8a68f3fb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:07.787617 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110daa27-1a48-4d1f-a4ad-beb8a68f3fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:07.788476 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110daa27-1a48-4d1f-a4ad-beb8a68f3fb4 admin admin] PATH_INFO: `/v3/projects/600bf1fbb9134fcaaa38e2432c980d75/users/b069eb25795e448598cb9723bcc05c6e/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:07.852385 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110daa27-1a48-4d1f-a4ad-beb8a68f3fb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=600bf1fbb9134fcaaa38e2432c980d75, user_id=b069eb25795e448598cb9723bcc05c6e, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:07.855504 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110daa27-1a48-4d1f-a4ad-beb8a68f3fb4 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:07.888193 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 902/1791] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:47:07 2022] PUT /identity/v3/projects/600bf1fbb9134fcaaa38e2432c980d75/users/b069eb25795e448598cb9723bcc05c6e/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:47:07.892447 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fecfc42-2049-4bf9-a236-2ceffcc975b9 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:07.927833 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fecfc42-2049-4bf9-a236-2ceffcc975b9 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:08.128470 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:DELETE=1,keystone:SELECT=95,keystone:INSERT=4 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:08.166579 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110daa27-1a48-4d1f-a4ad-beb8a68f3fb4 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:08.167681 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a69a81-a785-481e-ba5c-c8266bee6100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:08.168695 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-110daa27-1a48-4d1f-a4ad-beb8a68f3fb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:08.169411 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a69a81-a785-481e-ba5c-c8266bee6100 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:08.171130 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a69a81-a785-481e-ba5c-c8266bee6100 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:08.172394 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a69a81-a785-481e-ba5c-c8266bee6100 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:08.174275 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a69a81-a785-481e-ba5c-c8266bee6100 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:08.176790 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a69a81-a785-481e-ba5c-c8266bee6100 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:08.200495 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110daa27-1a48-4d1f-a4ad-beb8a68f3fb4 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:08.208737 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 890/1792] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:47:07 2022] GET /identity/v3/roles => generated 1420 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:47:08.208737 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a69a81-a785-481e-ba5c-c8266bee6100 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:08.212091 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-86a69a81-a785-481e-ba5c-c8266bee6100 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:08.224529 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fccfd5-6fd8-4697-b5da-c25e78049bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:08.225631 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fccfd5-6fd8-4697-b5da-c25e78049bf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:08.226025 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fccfd5-6fd8-4697-b5da-c25e78049bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:08.226401 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fccfd5-6fd8-4697-b5da-c25e78049bf1 admin admin] PATH_INFO: `/v3/projects/495be81242fd48ca97f6353d69d54dca` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:08.229995 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fccfd5-6fd8-4697-b5da-c25e78049bf1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=495be81242fd48ca97f6353d69d54dca)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:08.235719 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fccfd5-6fd8-4697-b5da-c25e78049bf1 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:08.255920 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a69a81-a785-481e-ba5c-c8266bee6100 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:08.273896 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f06aa49-8271-478a-bee4-b6ea4362beb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:08.275104 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f06aa49-8271-478a-bee4-b6ea4362beb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:08.276826 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f06aa49-8271-478a-bee4-b6ea4362beb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:08.277125 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f06aa49-8271-478a-bee4-b6ea4362beb4 admin admin] PATH_INFO: `/v3/projects/c086fe2e5e8246e5b0077d8e283f3891` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:08.282537 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f06aa49-8271-478a-bee4-b6ea4362beb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c086fe2e5e8246e5b0077d8e283f3891)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:08.288575 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f06aa49-8271-478a-bee4-b6ea4362beb4 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:08.385156 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-91fccfd5-6fd8-4697-b5da-c25e78049bf1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '495be81242fd48ca97f6353d69d54dca', 'request_id': 'req-91fccfd5-6fd8-4697-b5da-c25e78049bf1'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:08.414545 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-91fccfd5-6fd8-4697-b5da-c25e78049bf1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '495be81242fd48ca97f6353d69d54dca', 'request_id': 'req-91fccfd5-6fd8-4697-b5da-c25e78049bf1'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:08.416590 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-5f06aa49-8271-478a-bee4-b6ea4362beb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c086fe2e5e8246e5b0077d8e283f3891', 'request_id': 'req-5f06aa49-8271-478a-bee4-b6ea4362beb4'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:08.428621 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-5f06aa49-8271-478a-bee4-b6ea4362beb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c086fe2e5e8246e5b0077d8e283f3891', 'request_id': 'req-5f06aa49-8271-478a-bee4-b6ea4362beb4'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:08.440859 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-91fccfd5-6fd8-4697-b5da-c25e78049bf1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '495be81242fd48ca97f6353d69d54dca', 'request_id': 'req-91fccfd5-6fd8-4697-b5da-c25e78049bf1'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:08.446845 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-5f06aa49-8271-478a-bee4-b6ea4362beb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c086fe2e5e8246e5b0077d8e283f3891', 'request_id': 'req-5f06aa49-8271-478a-bee4-b6ea4362beb4'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:08.464501 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-91fccfd5-6fd8-4697-b5da-c25e78049bf1 admin admin] The token cache is being invalidate because project 495be81242fd48ca97f6353d69d54dca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:47:08.464807 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-91fccfd5-6fd8-4697-b5da-c25e78049bf1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:08.473175 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-91fccfd5-6fd8-4697-b5da-c25e78049bf1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:08.475738 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-5f06aa49-8271-478a-bee4-b6ea4362beb4 admin admin] The token cache is being invalidate because project c086fe2e5e8246e5b0077d8e283f3891 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90653) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:47:08.476220 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 903/1793] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:47:08 2022] DELETE /identity/v3/projects/495be81242fd48ca97f6353d69d54dca => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 10:47:08.477039 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fccfd5-6fd8-4697-b5da-c25e78049bf1 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:08.478421 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-91fccfd5-6fd8-4697-b5da-c25e78049bf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:08.481900 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-5f06aa49-8271-478a-bee4-b6ea4362beb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:08.484917 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-5f06aa49-8271-478a-bee4-b6ea4362beb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:08.491888 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 891/1794] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:47:08 2022] DELETE /identity/v3/projects/c086fe2e5e8246e5b0077d8e283f3891 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:47:08.494994 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f06aa49-8271-478a-bee4-b6ea4362beb4 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:08.497665 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-5f06aa49-8271-478a-bee4-b6ea4362beb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:08.615698 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:08.662149 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fccfd5-6fd8-4697-b5da-c25e78049bf1 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:08.669349 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa2ee76-2320-445b-a5f5-fdc24ef8629e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:08.670113 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa2ee76-2320-445b-a5f5-fdc24ef8629e admin admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:08.670503 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa2ee76-2320-445b-a5f5-fdc24ef8629e admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:08.671330 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa2ee76-2320-445b-a5f5-fdc24ef8629e admin admin] PATH_INFO: `/v3/projects/600bf1fbb9134fcaaa38e2432c980d75/users/b069eb25795e448598cb9723bcc05c6e/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:08.673875 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa2ee76-2320-445b-a5f5-fdc24ef8629e admin admin] RBAC: Authorizing `identity:create_grant(project_id=600bf1fbb9134fcaaa38e2432c980d75, user_id=b069eb25795e448598cb9723bcc05c6e, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:08.677668 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa2ee76-2320-445b-a5f5-fdc24ef8629e admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:08.683560 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f06aa49-8271-478a-bee4-b6ea4362beb4 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:08.696594 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0789a58-1cfe-40b9-8e4a-f11d6766836e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:08.700011 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0789a58-1cfe-40b9-8e4a-f11d6766836e admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:08.700346 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0789a58-1cfe-40b9-8e4a-f11d6766836e admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:08.700708 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0789a58-1cfe-40b9-8e4a-f11d6766836e admin admin] PATH_INFO: `/v3/users/23b8587a62174d6aa6951a8d64c1edb9` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:08.708465 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0789a58-1cfe-40b9-8e4a-f11d6766836e admin admin] RBAC: Authorizing `identity:delete_user(user_id=23b8587a62174d6aa6951a8d64c1edb9)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:08.709151 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 904/1795] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:47:08 2022] PUT /identity/v3/projects/600bf1fbb9134fcaaa38e2432c980d75/users/b069eb25795e448598cb9723bcc05c6e/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:47:08.710386 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa2ee76-2320-445b-a5f5-fdc24ef8629e admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:08.723142 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-aaa2ee76-2320-445b-a5f5-fdc24ef8629e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:08.723329 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0789a58-1cfe-40b9-8e4a-f11d6766836e admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:08.866024 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-e0789a58-1cfe-40b9-8e4a-f11d6766836e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23b8587a62174d6aa6951a8d64c1edb9'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:08.879267 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-e0789a58-1cfe-40b9-8e4a-f11d6766836e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23b8587a62174d6aa6951a8d64c1edb9'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:08.881534 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-e0789a58-1cfe-40b9-8e4a-f11d6766836e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23b8587a62174d6aa6951a8d64c1edb9'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:08.885620 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa2ee76-2320-445b-a5f5-fdc24ef8629e admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:08.887373 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-e0789a58-1cfe-40b9-8e4a-f11d6766836e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23b8587a62174d6aa6951a8d64c1edb9'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:08.893551 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9390926f-34bd-41dc-9f3f-d5458f84035d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:08.894802 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9390926f-34bd-41dc-9f3f-d5458f84035d admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:08.895358 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9390926f-34bd-41dc-9f3f-d5458f84035d admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:08.896113 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9390926f-34bd-41dc-9f3f-d5458f84035d admin admin] PATH_INFO: `/v3/users/435029a94a684f7a8ddb439996bc5220` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:08.897772 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9390926f-34bd-41dc-9f3f-d5458f84035d admin admin] RBAC: Authorizing `identity:delete_user(user_id=435029a94a684f7a8ddb439996bc5220)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:08.915881 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9390926f-34bd-41dc-9f3f-d5458f84035d admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:08.921544 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-e0789a58-1cfe-40b9-8e4a-f11d6766836e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23b8587a62174d6aa6951a8d64c1edb9'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:08.957933 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:08.999591 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 892/1796] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:47:08 2022] DELETE /identity/v3/users/23b8587a62174d6aa6951a8d64c1edb9 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:47:09.003733 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c35b54d-a883-44c6-8381-a8565f409a90 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:09.003733 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c35b54d-a883-44c6-8381-a8565f409a90 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:09.003733 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c35b54d-a883-44c6-8381-a8565f409a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:09.037472 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-9390926f-34bd-41dc-9f3f-d5458f84035d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '435029a94a684f7a8ddb439996bc5220'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:09.056235 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-9390926f-34bd-41dc-9f3f-d5458f84035d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '435029a94a684f7a8ddb439996bc5220'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:09.057858 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-9390926f-34bd-41dc-9f3f-d5458f84035d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '435029a94a684f7a8ddb439996bc5220'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:09.058888 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-9390926f-34bd-41dc-9f3f-d5458f84035d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '435029a94a684f7a8ddb439996bc5220'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:09.072633 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-3c35b54d-a883-44c6-8381-a8565f409a90 None None] MFA Rules not processed for user `b069eb25795e448598cb9723bcc05c6e`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:47:09.073839 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-3c35b54d-a883-44c6-8381-a8565f409a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:09.114334 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-9390926f-34bd-41dc-9f3f-d5458f84035d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '435029a94a684f7a8ddb439996bc5220'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:09.163752 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 905/1797] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:47:08 2022] DELETE /identity/v3/users/435029a94a684f7a8ddb439996bc5220 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:47:09.272965 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 893/1798] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:47:08 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:47:09.286842 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9390926f-34bd-41dc-9f3f-d5458f84035d admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:09.291774 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-9390926f-34bd-41dc-9f3f-d5458f84035d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:09.473115 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9390926f-34bd-41dc-9f3f-d5458f84035d admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:09.492876 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec373d0b-e219-42f3-860c-3cb1517a2e88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:09.494070 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec373d0b-e219-42f3-860c-3cb1517a2e88 service cinder] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:09.494779 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec373d0b-e219-42f3-860c-3cb1517a2e88 service cinder] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:09.512600 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec373d0b-e219-42f3-860c-3cb1517a2e88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:09.535676 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:09.571971 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c35b54d-a883-44c6-8381-a8565f409a90 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:09.573599 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-3c35b54d-a883-44c6-8381-a8565f409a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:09.665576 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec373d0b-e219-42f3-860c-3cb1517a2e88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:09.669102 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec373d0b-e219-42f3-860c-3cb1517a2e88 service cinder] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:09.699701 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 906/1799] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:47:09 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:47:09.832655 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c35b54d-a883-44c6-8381-a8565f409a90 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:10.088961 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6596f722-e181-48f3-8e72-3e671bcb9c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:10.090088 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6596f722-e181-48f3-8e72-3e671bcb9c32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:10.090088 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6596f722-e181-48f3-8e72-3e671bcb9c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:10.090265 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6596f722-e181-48f3-8e72-3e671bcb9c32 admin admin] PATH_INFO: `/v3/projects/a37be821dec241f29ddba0a7b68f2801` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:10.100471 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6596f722-e181-48f3-8e72-3e671bcb9c32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a37be821dec241f29ddba0a7b68f2801)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:10.102929 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6596f722-e181-48f3-8e72-3e671bcb9c32 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:10.178783 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:10.219025 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-6596f722-e181-48f3-8e72-3e671bcb9c32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a37be821dec241f29ddba0a7b68f2801', 'request_id': 'req-6596f722-e181-48f3-8e72-3e671bcb9c32'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:10.238350 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-6596f722-e181-48f3-8e72-3e671bcb9c32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a37be821dec241f29ddba0a7b68f2801', 'request_id': 'req-6596f722-e181-48f3-8e72-3e671bcb9c32'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:10.267963 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-6596f722-e181-48f3-8e72-3e671bcb9c32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a37be821dec241f29ddba0a7b68f2801', 'request_id': 'req-6596f722-e181-48f3-8e72-3e671bcb9c32'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:10.302063 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-6596f722-e181-48f3-8e72-3e671bcb9c32 admin admin] The token cache is being invalidate because project a37be821dec241f29ddba0a7b68f2801 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90653) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:47:10.302604 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-6596f722-e181-48f3-8e72-3e671bcb9c32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:10.308987 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-6596f722-e181-48f3-8e72-3e671bcb9c32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:10.311488 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 894/1800] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:47:09 2022] DELETE /identity/v3/projects/a37be821dec241f29ddba0a7b68f2801 => generated 0 bytes in 740 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 10:47:10.316660 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec373d0b-e219-42f3-860c-3cb1517a2e88 service cinder] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:10.343507 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec373d0b-e219-42f3-860c-3cb1517a2e88 service cinder] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:10.545071 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76236cb3-7e51-42b8-b086-1efdd6967430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:10.545825 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76236cb3-7e51-42b8-b086-1efdd6967430 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:10.546115 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76236cb3-7e51-42b8-b086-1efdd6967430 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:10.546347 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76236cb3-7e51-42b8-b086-1efdd6967430 admin admin] PATH_INFO: `/v3/users/a8b09c040e3c41a6806564565cc994ef` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:10.547583 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76236cb3-7e51-42b8-b086-1efdd6967430 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8b09c040e3c41a6806564565cc994ef)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:10.558592 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76236cb3-7e51-42b8-b086-1efdd6967430 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:10.651082 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6596f722-e181-48f3-8e72-3e671bcb9c32 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:10.653616 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-6596f722-e181-48f3-8e72-3e671bcb9c32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:10.663110 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-76236cb3-7e51-42b8-b086-1efdd6967430 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8b09c040e3c41a6806564565cc994ef'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:10.675170 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-76236cb3-7e51-42b8-b086-1efdd6967430 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8b09c040e3c41a6806564565cc994ef'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:10.676290 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-76236cb3-7e51-42b8-b086-1efdd6967430 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8b09c040e3c41a6806564565cc994ef'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:10.677458 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-76236cb3-7e51-42b8-b086-1efdd6967430 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8b09c040e3c41a6806564565cc994ef'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:10.679060 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6596f722-e181-48f3-8e72-3e671bcb9c32 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:10.689164 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b27d2af-ea65-40d1-8186-73baebf96ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:10.690882 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27d2af-ea65-40d1-8186-73baebf96ada admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:10.691473 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27d2af-ea65-40d1-8186-73baebf96ada admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:10.691952 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27d2af-ea65-40d1-8186-73baebf96ada admin admin] PATH_INFO: `/v3/users/2856de2342bc4bc9a9fa91f1cb7aa1fb` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:10.693916 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b27d2af-ea65-40d1-8186-73baebf96ada admin admin] RBAC: Authorizing `identity:delete_user(user_id=2856de2342bc4bc9a9fa91f1cb7aa1fb)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:10.700967 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b27d2af-ea65-40d1-8186-73baebf96ada admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:10.723853 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-76236cb3-7e51-42b8-b086-1efdd6967430 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8b09c040e3c41a6806564565cc994ef'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:10.773464 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 907/1801] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:47:10 2022] DELETE /identity/v3/users/a8b09c040e3c41a6806564565cc994ef => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:47:10.829727 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-1b27d2af-ea65-40d1-8186-73baebf96ada admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2856de2342bc4bc9a9fa91f1cb7aa1fb'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:10.859141 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-1b27d2af-ea65-40d1-8186-73baebf96ada admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2856de2342bc4bc9a9fa91f1cb7aa1fb'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:10.861276 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-1b27d2af-ea65-40d1-8186-73baebf96ada admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2856de2342bc4bc9a9fa91f1cb7aa1fb'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:10.864840 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-1b27d2af-ea65-40d1-8186-73baebf96ada admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2856de2342bc4bc9a9fa91f1cb7aa1fb'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:10.888927 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-1b27d2af-ea65-40d1-8186-73baebf96ada admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2856de2342bc4bc9a9fa91f1cb7aa1fb'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:10.939655 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 895/1802] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:47:10 2022] DELETE /identity/v3/users/2856de2342bc4bc9a9fa91f1cb7aa1fb => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:47:10.942057 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad553d7-4d03-4c23-88b7-a0d804f1f434 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:10.942349 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad553d7-4d03-4c23-88b7-a0d804f1f434 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:10.942651 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad553d7-4d03-4c23-88b7-a0d804f1f434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:10.993933 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-1ad553d7-4d03-4c23-88b7-a0d804f1f434 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:47:10.997655 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-1ad553d7-4d03-4c23-88b7-a0d804f1f434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:11.077115 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:11.212804 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 908/1803] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:47:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:47:11.355868 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b27d2af-ea65-40d1-8186-73baebf96ada admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:11.357593 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-1b27d2af-ea65-40d1-8186-73baebf96ada admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:11.514742 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b27d2af-ea65-40d1-8186-73baebf96ada admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:11.520568 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9049b7-c30f-41e0-815c-741fdab0e31d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:11.521601 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9049b7-c30f-41e0-815c-741fdab0e31d service neutron] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:11.522149 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9049b7-c30f-41e0-815c-741fdab0e31d service neutron] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:11.522867 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9049b7-c30f-41e0-815c-741fdab0e31d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:11.535011 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9049b7-c30f-41e0-815c-741fdab0e31d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:11.537003 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9049b7-c30f-41e0-815c-741fdab0e31d service neutron] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:11.549127 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 896/1804] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:47:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:47:11.966839 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad553d7-4d03-4c23-88b7-a0d804f1f434 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:11.968531 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-1ad553d7-4d03-4c23-88b7-a0d804f1f434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:11.989617 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad553d7-4d03-4c23-88b7-a0d804f1f434 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:12.006014 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2b7dc3-770d-4d5d-98eb-6a9f2258423c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:12.006739 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b7dc3-770d-4d5d-98eb-6a9f2258423c admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:12.007021 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b7dc3-770d-4d5d-98eb-6a9f2258423c admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:12.007506 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b7dc3-770d-4d5d-98eb-6a9f2258423c admin admin] PATH_INFO: `/v3/projects/7ae4b30a9e304c92ae0185cfea78f4e3` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:12.010271 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2b7dc3-770d-4d5d-98eb-6a9f2258423c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ae4b30a9e304c92ae0185cfea78f4e3)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:12.013861 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2b7dc3-770d-4d5d-98eb-6a9f2258423c admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:12.145264 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-ed2b7dc3-770d-4d5d-98eb-6a9f2258423c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ae4b30a9e304c92ae0185cfea78f4e3', 'request_id': 'req-ed2b7dc3-770d-4d5d-98eb-6a9f2258423c'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:12.162896 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-ed2b7dc3-770d-4d5d-98eb-6a9f2258423c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ae4b30a9e304c92ae0185cfea78f4e3', 'request_id': 'req-ed2b7dc3-770d-4d5d-98eb-6a9f2258423c'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:12.190724 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-ed2b7dc3-770d-4d5d-98eb-6a9f2258423c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ae4b30a9e304c92ae0185cfea78f4e3', 'request_id': 'req-ed2b7dc3-770d-4d5d-98eb-6a9f2258423c'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:12.223906 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-ed2b7dc3-770d-4d5d-98eb-6a9f2258423c admin admin] The token cache is being invalidate because project 7ae4b30a9e304c92ae0185cfea78f4e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:47:12.224557 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-ed2b7dc3-770d-4d5d-98eb-6a9f2258423c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:12.225524 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-ed2b7dc3-770d-4d5d-98eb-6a9f2258423c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:12.228760 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 909/1805] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:47:11 2022] DELETE /identity/v3/projects/7ae4b30a9e304c92ae0185cfea78f4e3 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:47:12.236240 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee59b88d-d1ba-4edc-8160-e2f17740fd5b None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:12.237946 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee59b88d-d1ba-4edc-8160-e2f17740fd5b None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:12.238451 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee59b88d-d1ba-4edc-8160-e2f17740fd5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:12.271985 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-ee59b88d-d1ba-4edc-8160-e2f17740fd5b None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:47:12.273711 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-ee59b88d-d1ba-4edc-8160-e2f17740fd5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:12.312258 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:12.419236 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 897/1806] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:47:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:47:12.421076 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8513c236-5190-45f6-9f61-ab5ba7dbc037 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:12.421456 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8513c236-5190-45f6-9f61-ab5ba7dbc037 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:12.421688 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8513c236-5190-45f6-9f61-ab5ba7dbc037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:12.454183 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-8513c236-5190-45f6-9f61-ab5ba7dbc037 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:47:12.457503 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-8513c236-5190-45f6-9f61-ab5ba7dbc037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:12.476644 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 910/1807] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:47:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:47:12.481030 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee59b88d-d1ba-4edc-8160-e2f17740fd5b None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:12.498874 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee59b88d-d1ba-4edc-8160-e2f17740fd5b None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:12.518750 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e079c90-3085-4599-a145-cc02825a1542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:12.519862 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e079c90-3085-4599-a145-cc02825a1542 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:12.520366 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e079c90-3085-4599-a145-cc02825a1542 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:12.520701 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e079c90-3085-4599-a145-cc02825a1542 admin admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:12.521491 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e079c90-3085-4599-a145-cc02825a1542 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:47:12.521930 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e079c90-3085-4599-a145-cc02825a1542 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:12.527320 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e079c90-3085-4599-a145-cc02825a1542 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:12.569843 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 898/1808] 10.222.0.15 () {58 vars in 1153 bytes} [Sun Sep 11 10:47:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:47:12.573858 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8513c236-5190-45f6-9f61-ab5ba7dbc037 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:12.593789 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8513c236-5190-45f6-9f61-ab5ba7dbc037 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:12.609680 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b701f9c-50a1-46be-958e-5eba2d386576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:12.610340 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b701f9c-50a1-46be-958e-5eba2d386576 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:12.610622 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b701f9c-50a1-46be-958e-5eba2d386576 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:12.610936 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b701f9c-50a1-46be-958e-5eba2d386576 admin admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:12.612439 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b701f9c-50a1-46be-958e-5eba2d386576 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:12.614760 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b701f9c-50a1-46be-958e-5eba2d386576 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:12.674913 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 911/1809] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:47:12 2022] POST /identity/v3/projects => generated 367 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:47:12.678969 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e079c90-3085-4599-a145-cc02825a1542 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:12.697157 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e079c90-3085-4599-a145-cc02825a1542 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:12.857056 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a27be95-cbd5-4148-807a-46431a5d7746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:12.858599 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a27be95-cbd5-4148-807a-46431a5d7746 admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:12.858885 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a27be95-cbd5-4148-807a-46431a5d7746 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:12.859159 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a27be95-cbd5-4148-807a-46431a5d7746 admin admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:12.859923 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a27be95-cbd5-4148-807a-46431a5d7746 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:12.864999 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a27be95-cbd5-4148-807a-46431a5d7746 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:12.914434 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 899/1810] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:47:12 2022] POST /identity/v3/users => generated 284 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:47:12.922381 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b701f9c-50a1-46be-958e-5eba2d386576 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:12.942711 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b701f9c-50a1-46be-958e-5eba2d386576 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:12.951382 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5d0cc0-5e5b-4632-ae0d-3911be6745f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:12.952055 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5d0cc0-5e5b-4632-ae0d-3911be6745f5 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:12.952387 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5d0cc0-5e5b-4632-ae0d-3911be6745f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:12.952665 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5d0cc0-5e5b-4632-ae0d-3911be6745f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:12.953387 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5d0cc0-5e5b-4632-ae0d-3911be6745f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:12.954999 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5d0cc0-5e5b-4632-ae0d-3911be6745f5 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:12.971941 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 912/1811] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:47:12 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:47:12.976311 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a27be95-cbd5-4148-807a-46431a5d7746 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:12.995295 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a27be95-cbd5-4148-807a-46431a5d7746 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:13.003062 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7529d0-3abf-4bbb-a31f-90392da75393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:13.003940 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7529d0-3abf-4bbb-a31f-90392da75393 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:13.004347 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7529d0-3abf-4bbb-a31f-90392da75393 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:13.004662 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7529d0-3abf-4bbb-a31f-90392da75393 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:13.005573 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7529d0-3abf-4bbb-a31f-90392da75393 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:13.019449 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7529d0-3abf-4bbb-a31f-90392da75393 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:13.028515 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 900/1812] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:47:12 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:47:13.034315 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5d0cc0-5e5b-4632-ae0d-3911be6745f5 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:13.049117 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5d0cc0-5e5b-4632-ae0d-3911be6745f5 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:13.056047 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583b0ccb-7889-4b7c-8fcd-82dda881d139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:13.056794 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583b0ccb-7889-4b7c-8fcd-82dda881d139 admin admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:13.057120 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583b0ccb-7889-4b7c-8fcd-82dda881d139 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:13.057500 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583b0ccb-7889-4b7c-8fcd-82dda881d139 admin admin] PATH_INFO: `/v3/projects/57a8607224b74110ad15ca9a8deacabe/users/d4fa5cef363b406da7a207f4e4bf3ccc/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:13.094844 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583b0ccb-7889-4b7c-8fcd-82dda881d139 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57a8607224b74110ad15ca9a8deacabe, user_id=d4fa5cef363b406da7a207f4e4bf3ccc, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:13.101693 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583b0ccb-7889-4b7c-8fcd-82dda881d139 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:13.135029 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 913/1813] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:47:13 2022] PUT /identity/v3/projects/57a8607224b74110ad15ca9a8deacabe/users/d4fa5cef363b406da7a207f4e4bf3ccc/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:47:13.153946 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7f535-7791-469b-9ca0-9b2508f442cf None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:13.154379 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7f535-7791-469b-9ca0-9b2508f442cf None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:13.154794 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7f535-7791-469b-9ca0-9b2508f442cf None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:13.159717 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 901/1814] 10.222.0.15 () {54 vars in 921 bytes} [Sun Sep 11 10:47:13 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:47:13.161296 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d7f535-7791-469b-9ca0-9b2508f442cf None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:13.165516 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-86d7f535-7791-469b-9ca0-9b2508f442cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:13.174644 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e4fa8-e19e-49b3-b37d-0c3f69abaf88 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:13.179770 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e4fa8-e19e-49b3-b37d-0c3f69abaf88 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:13.192122 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e4fa8-e19e-49b3-b37d-0c3f69abaf88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:13.286517 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-295e4fa8-e19e-49b3-b37d-0c3f69abaf88 None None] MFA Rules not processed for user `d4fa5cef363b406da7a207f4e4bf3ccc`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:47:13.287783 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-295e4fa8-e19e-49b3-b37d-0c3f69abaf88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:13.416742 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:13.436651 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d7f535-7791-469b-9ca0-9b2508f442cf None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:13.452651 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568ba566-5e53-4629-ae85-0ddb9586a999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:13.453379 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568ba566-5e53-4629-ae85-0ddb9586a999 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:13.453667 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568ba566-5e53-4629-ae85-0ddb9586a999 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:13.453942 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568ba566-5e53-4629-ae85-0ddb9586a999 admin admin] PATH_INFO: `/v3/projects/b97cf9e0fa6a4dd4a4f8222041030624` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:13.456706 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568ba566-5e53-4629-ae85-0ddb9586a999 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b97cf9e0fa6a4dd4a4f8222041030624)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:13.458701 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568ba566-5e53-4629-ae85-0ddb9586a999 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:13.470251 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 914/1815] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:47:13 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:47:13.473998 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ef80b3-ea66-4aae-a627-8b01e55bf1d2 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:13.474553 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ef80b3-ea66-4aae-a627-8b01e55bf1d2 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:13.476246 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ef80b3-ea66-4aae-a627-8b01e55bf1d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:13.508438 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-59ef80b3-ea66-4aae-a627-8b01e55bf1d2 None None] MFA Rules not processed for user `2bd5356829b6454993bab9bad2f79fed`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:47:13.510846 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-59ef80b3-ea66-4aae-a627-8b01e55bf1d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:13.559888 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:13.586958 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-568ba566-5e53-4629-ae85-0ddb9586a999 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b97cf9e0fa6a4dd4a4f8222041030624', 'request_id': 'req-568ba566-5e53-4629-ae85-0ddb9586a999'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:13.600230 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-568ba566-5e53-4629-ae85-0ddb9586a999 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b97cf9e0fa6a4dd4a4f8222041030624', 'request_id': 'req-568ba566-5e53-4629-ae85-0ddb9586a999'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:13.630449 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-568ba566-5e53-4629-ae85-0ddb9586a999 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b97cf9e0fa6a4dd4a4f8222041030624', 'request_id': 'req-568ba566-5e53-4629-ae85-0ddb9586a999'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:13.655829 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-568ba566-5e53-4629-ae85-0ddb9586a999 admin admin] The token cache is being invalidate because project b97cf9e0fa6a4dd4a4f8222041030624 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90653) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:47:13.656227 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-568ba566-5e53-4629-ae85-0ddb9586a999 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:13.657443 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-568ba566-5e53-4629-ae85-0ddb9586a999 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:13.661077 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 902/1816] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:47:13 2022] DELETE /identity/v3/projects/b97cf9e0fa6a4dd4a4f8222041030624 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:47:13.661646 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568ba566-5e53-4629-ae85-0ddb9586a999 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:13.665920 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-568ba566-5e53-4629-ae85-0ddb9586a999 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:13.708160 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 915/1817] 10.222.0.15 () {58 vars in 1018 bytes} [Sun Sep 11 10:47:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:47:13.710350 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ff4431-78d2-4d67-9cb2-88b0ac2d4af7 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:13.710882 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ff4431-78d2-4d67-9cb2-88b0ac2d4af7 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:13.711181 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ff4431-78d2-4d67-9cb2-88b0ac2d4af7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:13.805449 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-64ff4431-78d2-4d67-9cb2-88b0ac2d4af7 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:47:13.838544 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-64ff4431-78d2-4d67-9cb2-88b0ac2d4af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:13.952030 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568ba566-5e53-4629-ae85-0ddb9586a999 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:13.969400 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eec617-33f7-4ae5-8a2f-09c77a2b43a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:13.969959 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eec617-33f7-4ae5-8a2f-09c77a2b43a3 admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:13.970247 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eec617-33f7-4ae5-8a2f-09c77a2b43a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:13.970519 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eec617-33f7-4ae5-8a2f-09c77a2b43a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:13.971756 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eec617-33f7-4ae5-8a2f-09c77a2b43a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:13.973705 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eec617-33f7-4ae5-8a2f-09c77a2b43a3 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:14.036598 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 916/1818] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:47:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:47:14.037486 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ff4431-78d2-4d67-9cb2-88b0ac2d4af7 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:14.048657 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-64ff4431-78d2-4d67-9cb2-88b0ac2d4af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:14.060641 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 903/1819] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:47:13 2022] POST /identity/v3/projects => generated 365 bytes in 399 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:47:14.066490 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eca4cd-5a34-42f9-90be-a9bba2860ab9 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:14.066791 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eca4cd-5a34-42f9-90be-a9bba2860ab9 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:14.068035 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eca4cd-5a34-42f9-90be-a9bba2860ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:14.073941 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ff4431-78d2-4d67-9cb2-88b0ac2d4af7 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:14.136133 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-11eca4cd-5a34-42f9-90be-a9bba2860ab9 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:47:14.139083 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-11eca4cd-5a34-42f9-90be-a9bba2860ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:14.214357 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a48f1a2-15dd-41e0-902b-c4ccb1e72e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:14.215298 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a48f1a2-15dd-41e0-902b-c4ccb1e72e08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:14.218850 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a48f1a2-15dd-41e0-902b-c4ccb1e72e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:14.221947 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a48f1a2-15dd-41e0-902b-c4ccb1e72e08 admin admin] PATH_INFO: `/v3/projects/e093fa22c59b4cbd8498f0187067c598` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:14.227015 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a48f1a2-15dd-41e0-902b-c4ccb1e72e08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e093fa22c59b4cbd8498f0187067c598)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:14.230350 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a48f1a2-15dd-41e0-902b-c4ccb1e72e08 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:14.255001 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:14.272382 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 904/1820] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:47:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:47:14.273384 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eca4cd-5a34-42f9-90be-a9bba2860ab9 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:14.310085 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eca4cd-5a34-42f9-90be-a9bba2860ab9 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:14.344629 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-0a48f1a2-15dd-41e0-902b-c4ccb1e72e08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e093fa22c59b4cbd8498f0187067c598', 'request_id': 'req-0a48f1a2-15dd-41e0-902b-c4ccb1e72e08'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:14.362071 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-0a48f1a2-15dd-41e0-902b-c4ccb1e72e08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e093fa22c59b4cbd8498f0187067c598', 'request_id': 'req-0a48f1a2-15dd-41e0-902b-c4ccb1e72e08'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:14.374175 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-0a48f1a2-15dd-41e0-902b-c4ccb1e72e08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e093fa22c59b4cbd8498f0187067c598', 'request_id': 'req-0a48f1a2-15dd-41e0-902b-c4ccb1e72e08'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:14.398588 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-0a48f1a2-15dd-41e0-902b-c4ccb1e72e08 admin admin] The token cache is being invalidate because project e093fa22c59b4cbd8498f0187067c598 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:47:14.399058 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-0a48f1a2-15dd-41e0-902b-c4ccb1e72e08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:14.400135 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-0a48f1a2-15dd-41e0-902b-c4ccb1e72e08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:14.402185 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 917/1821] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:47:14 2022] DELETE /identity/v3/projects/e093fa22c59b4cbd8498f0187067c598 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:47:14.404689 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a48f1a2-15dd-41e0-902b-c4ccb1e72e08 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:14.406572 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-0a48f1a2-15dd-41e0-902b-c4ccb1e72e08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:14.422221 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:14.445641 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dd2abd-07d1-4bfc-b6fd-77ed4bea819d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:14.446352 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd2abd-07d1-4bfc-b6fd-77ed4bea819d admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:14.446522 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd2abd-07d1-4bfc-b6fd-77ed4bea819d admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:14.446803 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd2abd-07d1-4bfc-b6fd-77ed4bea819d admin admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:14.447455 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dd2abd-07d1-4bfc-b6fd-77ed4bea819d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:14.452380 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dd2abd-07d1-4bfc-b6fd-77ed4bea819d admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:14.501588 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 905/1822] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:47:14 2022] POST /identity/v3/users => generated 289 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:47:14.502517 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dd2abd-07d1-4bfc-b6fd-77ed4bea819d admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:14.504381 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-b2dd2abd-07d1-4bfc-b6fd-77ed4bea819d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:14.531294 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dd2abd-07d1-4bfc-b6fd-77ed4bea819d admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:14.535193 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a48f1a2-15dd-41e0-902b-c4ccb1e72e08 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:14.538314 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caf72f5-7d1d-4642-a846-84e3fed0eea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:14.539270 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf72f5-7d1d-4642-a846-84e3fed0eea1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:14.539889 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf72f5-7d1d-4642-a846-84e3fed0eea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:14.540409 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf72f5-7d1d-4642-a846-84e3fed0eea1 admin admin] PATH_INFO: `/v3/users/b069eb25795e448598cb9723bcc05c6e` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:14.541941 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caf72f5-7d1d-4642-a846-84e3fed0eea1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b069eb25795e448598cb9723bcc05c6e)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:14.542464 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2218efd-67c4-49cd-a0bd-5ae07ad9b4e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:14.543518 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2218efd-67c4-49cd-a0bd-5ae07ad9b4e8 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:14.544134 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2218efd-67c4-49cd-a0bd-5ae07ad9b4e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:14.544560 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2218efd-67c4-49cd-a0bd-5ae07ad9b4e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:14.544876 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caf72f5-7d1d-4642-a846-84e3fed0eea1 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:14.545616 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2218efd-67c4-49cd-a0bd-5ae07ad9b4e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:47:14.546162 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2218efd-67c4-49cd-a0bd-5ae07ad9b4e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:14.549466 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2218efd-67c4-49cd-a0bd-5ae07ad9b4e8 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:14.566903 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 918/1823] 10.222.0.15 () {58 vars in 1153 bytes} [Sun Sep 11 10:47:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:47:14.569538 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9c4dc1-5f97-4e2d-a2b0-6d77fbd61fda None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:14.569826 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9c4dc1-5f97-4e2d-a2b0-6d77fbd61fda None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:14.570219 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9c4dc1-5f97-4e2d-a2b0-6d77fbd61fda None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:14.572677 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 919/1824] 10.222.0.15 () {54 vars in 921 bytes} [Sun Sep 11 10:47:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:47:14.573575 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9c4dc1-5f97-4e2d-a2b0-6d77fbd61fda None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:14.575888 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-6e9c4dc1-5f97-4e2d-a2b0-6d77fbd61fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:14.620633 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9c4dc1-5f97-4e2d-a2b0-6d77fbd61fda None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:14.626900 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e61734-66a9-49b3-8fbe-4b347564f94e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:14.627762 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e61734-66a9-49b3-8fbe-4b347564f94e admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:14.628544 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e61734-66a9-49b3-8fbe-4b347564f94e admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:14.628918 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e61734-66a9-49b3-8fbe-4b347564f94e admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:14.629640 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e61734-66a9-49b3-8fbe-4b347564f94e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:14.631631 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e61734-66a9-49b3-8fbe-4b347564f94e admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:14.649957 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 920/1825] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:47:14 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:47:14.651026 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e61734-66a9-49b3-8fbe-4b347564f94e admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:14.666464 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-4caf72f5-7d1d-4642-a846-84e3fed0eea1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b069eb25795e448598cb9723bcc05c6e'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:14.667077 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e61734-66a9-49b3-8fbe-4b347564f94e admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:14.675235 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e152de62-da67-4fbc-8322-6aa47c6bcde2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:14.676029 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e152de62-da67-4fbc-8322-6aa47c6bcde2 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:14.676318 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e152de62-da67-4fbc-8322-6aa47c6bcde2 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:14.676590 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e152de62-da67-4fbc-8322-6aa47c6bcde2 admin admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:14.677280 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e152de62-da67-4fbc-8322-6aa47c6bcde2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:14.678567 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-4caf72f5-7d1d-4642-a846-84e3fed0eea1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b069eb25795e448598cb9723bcc05c6e'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:14.679494 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e152de62-da67-4fbc-8322-6aa47c6bcde2 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:14.679878 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-4caf72f5-7d1d-4642-a846-84e3fed0eea1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b069eb25795e448598cb9723bcc05c6e'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:14.680706 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-4caf72f5-7d1d-4642-a846-84e3fed0eea1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b069eb25795e448598cb9723bcc05c6e'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:14.691174 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-4caf72f5-7d1d-4642-a846-84e3fed0eea1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b069eb25795e448598cb9723bcc05c6e'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:14.729924 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 921/1826] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:47:14 2022] POST /identity/v3/projects => generated 383 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:47:14.731106 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e152de62-da67-4fbc-8322-6aa47c6bcde2 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:14.733544 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-e152de62-da67-4fbc-8322-6aa47c6bcde2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:14.739227 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 906/1827] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:47:14 2022] DELETE /identity/v3/users/b069eb25795e448598cb9723bcc05c6e => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:47:14.740346 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caf72f5-7d1d-4642-a846-84e3fed0eea1 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:14.742636 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-4caf72f5-7d1d-4642-a846-84e3fed0eea1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:14.865319 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:14.902057 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e152de62-da67-4fbc-8322-6aa47c6bcde2 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:14.930226 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caf72f5-7d1d-4642-a846-84e3fed0eea1 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:14.937567 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9519e74-f5a0-4817-9c3e-169c0f93de27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:14.938764 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9519e74-f5a0-4817-9c3e-169c0f93de27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:14.939277 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9519e74-f5a0-4817-9c3e-169c0f93de27 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:14.939810 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9519e74-f5a0-4817-9c3e-169c0f93de27 admin admin] PATH_INFO: `/v3/projects/b3bd5059c53941c0bc4deba0fa92cce1` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:14.942632 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9519e74-f5a0-4817-9c3e-169c0f93de27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3bd5059c53941c0bc4deba0fa92cce1)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:14.948429 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9519e74-f5a0-4817-9c3e-169c0f93de27 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:15.088540 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad39fd80-991d-411d-9c2a-2e90346a5a99 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bd5356829b6454993bab9bad2f79fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:15.089368 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad39fd80-991d-411d-9c2a-2e90346a5a99 service nova] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:15.089636 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad39fd80-991d-411d-9c2a-2e90346a5a99 service nova] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:15.091502 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad39fd80-991d-411d-9c2a-2e90346a5a99 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:15.092462 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-b9519e74-f5a0-4817-9c3e-169c0f93de27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3bd5059c53941c0bc4deba0fa92cce1', 'request_id': 'req-b9519e74-f5a0-4817-9c3e-169c0f93de27'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:15.096452 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=82,keystone:INSERT=4,keystone:DELETE=14 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:15.098825 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-ad39fd80-991d-411d-9c2a-2e90346a5a99 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:15.103363 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-b9519e74-f5a0-4817-9c3e-169c0f93de27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3bd5059c53941c0bc4deba0fa92cce1', 'request_id': 'req-b9519e74-f5a0-4817-9c3e-169c0f93de27'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:15.116551 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-b9519e74-f5a0-4817-9c3e-169c0f93de27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3bd5059c53941c0bc4deba0fa92cce1', 'request_id': 'req-b9519e74-f5a0-4817-9c3e-169c0f93de27'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:15.130864 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad39fd80-991d-411d-9c2a-2e90346a5a99 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:15.132518 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad39fd80-991d-411d-9c2a-2e90346a5a99 service nova] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:15.142552 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-b9519e74-f5a0-4817-9c3e-169c0f93de27 admin admin] The token cache is being invalidate because project b3bd5059c53941c0bc4deba0fa92cce1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90653) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:47:15.143266 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-b9519e74-f5a0-4817-9c3e-169c0f93de27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:15.144834 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-b9519e74-f5a0-4817-9c3e-169c0f93de27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:15.147334 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 907/1828] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:47:14 2022] DELETE /identity/v3/projects/b3bd5059c53941c0bc4deba0fa92cce1 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:47:15.149768 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62457a5f-b336-4b91-9389-322b3df370dd None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:15.150573 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62457a5f-b336-4b91-9389-322b3df370dd None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:15.151502 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62457a5f-b336-4b91-9389-322b3df370dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:15.157539 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 922/1829] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:47:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:47:15.158239 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad39fd80-991d-411d-9c2a-2e90346a5a99 service nova] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:15.166176 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-ad39fd80-991d-411d-9c2a-2e90346a5a99 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:15.198690 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad39fd80-991d-411d-9c2a-2e90346a5a99 service nova] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:15.203093 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-62457a5f-b336-4b91-9389-322b3df370dd None None] MFA Rules not processed for user `2bd5356829b6454993bab9bad2f79fed`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:47:15.206730 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-62457a5f-b336-4b91-9389-322b3df370dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:15.278498 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d785ab-9561-4cee-94f6-031f1190c116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:15.279377 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d785ab-9561-4cee-94f6-031f1190c116 admin admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:15.280124 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d785ab-9561-4cee-94f6-031f1190c116 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:15.280124 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d785ab-9561-4cee-94f6-031f1190c116 admin admin] PATH_INFO: `/v3/projects/44405dd11c564907a56ba7be0e027768/users/2986e5cce18d49cbb8156e98f5cf0725/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:15.298697 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d785ab-9561-4cee-94f6-031f1190c116 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44405dd11c564907a56ba7be0e027768, user_id=2986e5cce18d49cbb8156e98f5cf0725, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:15.302146 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d785ab-9561-4cee-94f6-031f1190c116 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:15.326076 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 923/1830] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:47:15 2022] PUT /identity/v3/projects/44405dd11c564907a56ba7be0e027768/users/2986e5cce18d49cbb8156e98f5cf0725/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:47:15.327541 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d785ab-9561-4cee-94f6-031f1190c116 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:15.329621 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-c2d785ab-9561-4cee-94f6-031f1190c116 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:15.365677 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 908/1831] 10.222.0.15 () {58 vars in 1018 bytes} [Sun Sep 11 10:47:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:47:15.367409 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9cd8f8-77da-4fc7-83e5-33e47c0d20b8 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:15.367562 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9cd8f8-77da-4fc7-83e5-33e47c0d20b8 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:15.367926 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9cd8f8-77da-4fc7-83e5-33e47c0d20b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:15.400980 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-cb9cd8f8-77da-4fc7-83e5-33e47c0d20b8 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:47:15.402381 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-cb9cd8f8-77da-4fc7-83e5-33e47c0d20b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:15.436060 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d785ab-9561-4cee-94f6-031f1190c116 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:15.445470 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289ceb89-9aff-483b-b732-deecde0de246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:15.446406 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ceb89-9aff-483b-b732-deecde0de246 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:15.447026 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ceb89-9aff-483b-b732-deecde0de246 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:15.448284 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ceb89-9aff-483b-b732-deecde0de246 admin admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:15.451760 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289ceb89-9aff-483b-b732-deecde0de246 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:15.455679 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289ceb89-9aff-483b-b732-deecde0de246 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:15.479614 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 909/1832] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:47:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:47:15.479614 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9cd8f8-77da-4fc7-83e5-33e47c0d20b8 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:15.479614 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-cb9cd8f8-77da-4fc7-83e5-33e47c0d20b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:15.489950 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9cd8f8-77da-4fc7-83e5-33e47c0d20b8 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:15.491291 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:15.496980 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4077891e-37e9-4652-af5f-6e79e9264dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:15.497788 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4077891e-37e9-4652-af5f-6e79e9264dd1 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:15.498316 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4077891e-37e9-4652-af5f-6e79e9264dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:15.498667 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4077891e-37e9-4652-af5f-6e79e9264dd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:15.499852 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4077891e-37e9-4652-af5f-6e79e9264dd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:15.501568 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4077891e-37e9-4652-af5f-6e79e9264dd1 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:15.507415 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 924/1833] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:47:15 2022] POST /identity/v3/users => generated 292 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:47:15.512650 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce299ef4-631f-4f53-bbeb-3db729993346 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:15.513337 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce299ef4-631f-4f53-bbeb-3db729993346 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:15.513773 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce299ef4-631f-4f53-bbeb-3db729993346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:15.519604 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 910/1834] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:47:15 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:47:15.520894 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4077891e-37e9-4652-af5f-6e79e9264dd1 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:15.533796 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4077891e-37e9-4652-af5f-6e79e9264dd1 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:15.542825 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bf4ed8-c5c7-459a-be21-53cb7b25493c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:15.543781 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bf4ed8-c5c7-459a-be21-53cb7b25493c admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:15.544056 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bf4ed8-c5c7-459a-be21-53cb7b25493c admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:15.544335 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bf4ed8-c5c7-459a-be21-53cb7b25493c admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:15.544989 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bf4ed8-c5c7-459a-be21-53cb7b25493c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:15.546669 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bf4ed8-c5c7-459a-be21-53cb7b25493c admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:15.548480 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-ce299ef4-631f-4f53-bbeb-3db729993346 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:47:15.550487 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-ce299ef4-631f-4f53-bbeb-3db729993346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:15.564200 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 925/1835] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:47:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:47:15.564937 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce299ef4-631f-4f53-bbeb-3db729993346 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:15.566738 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 911/1836] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:47:15 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:47:15.567580 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bf4ed8-c5c7-459a-be21-53cb7b25493c admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:15.576413 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce299ef4-631f-4f53-bbeb-3db729993346 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:15.581997 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c60a6a1-2d3c-4a1e-b296-e4f09016f502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:15.582584 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c60a6a1-2d3c-4a1e-b296-e4f09016f502 admin admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:15.583083 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c60a6a1-2d3c-4a1e-b296-e4f09016f502 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:15.583368 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c60a6a1-2d3c-4a1e-b296-e4f09016f502 admin admin] PATH_INFO: `/v3/projects/44405dd11c564907a56ba7be0e027768/users/2986e5cce18d49cbb8156e98f5cf0725/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:15.584125 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bf4ed8-c5c7-459a-be21-53cb7b25493c admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:15.585946 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c60a6a1-2d3c-4a1e-b296-e4f09016f502 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44405dd11c564907a56ba7be0e027768, user_id=2986e5cce18d49cbb8156e98f5cf0725, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:15.589262 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c60a6a1-2d3c-4a1e-b296-e4f09016f502 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:15.590491 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368e7ec5-8e1c-485a-bee7-25af345d60ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:15.591639 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e7ec5-8e1c-485a-bee7-25af345d60ca admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:15.591937 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e7ec5-8e1c-485a-bee7-25af345d60ca admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:15.592214 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e7ec5-8e1c-485a-bee7-25af345d60ca admin admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:15.593253 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368e7ec5-8e1c-485a-bee7-25af345d60ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:47:15.593584 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368e7ec5-8e1c-485a-bee7-25af345d60ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:15.595748 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368e7ec5-8e1c-485a-bee7-25af345d60ca admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:15.612693 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 926/1837] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:47:15 2022] PUT /identity/v3/projects/44405dd11c564907a56ba7be0e027768/users/2986e5cce18d49cbb8156e98f5cf0725/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:47:15.614482 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c60a6a1-2d3c-4a1e-b296-e4f09016f502 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:15.621936 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 912/1838] 10.222.0.15 () {58 vars in 1153 bytes} [Sun Sep 11 10:47:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:47:15.626104 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d932f-1e64-4baa-b3eb-52296acf6604 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:15.626577 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d932f-1e64-4baa-b3eb-52296acf6604 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:15.627040 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d932f-1e64-4baa-b3eb-52296acf6604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:15.677858 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-ee1d932f-1e64-4baa-b3eb-52296acf6604 None None] MFA Rules not processed for user `2986e5cce18d49cbb8156e98f5cf0725`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:47:15.682015 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-ee1d932f-1e64-4baa-b3eb-52296acf6604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:15.714531 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c60a6a1-2d3c-4a1e-b296-e4f09016f502 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:15.719997 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d1a6b7-1f20-4ada-b296-ccc36de04a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:15.720624 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d1a6b7-1f20-4ada-b296-ccc36de04a34 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:15.721438 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d1a6b7-1f20-4ada-b296-ccc36de04a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:15.721716 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d1a6b7-1f20-4ada-b296-ccc36de04a34 admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:15.722425 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d1a6b7-1f20-4ada-b296-ccc36de04a34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:15.723656 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:15.725337 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d1a6b7-1f20-4ada-b296-ccc36de04a34 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:15.744232 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 927/1839] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:47:15 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:47:15.745532 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d1a6b7-1f20-4ada-b296-ccc36de04a34 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:15.757083 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d1a6b7-1f20-4ada-b296-ccc36de04a34 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:15.762782 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec785391-9afb-4393-9dc2-b0d614093020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:15.763849 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec785391-9afb-4393-9dc2-b0d614093020 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:15.764138 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec785391-9afb-4393-9dc2-b0d614093020 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:15.764415 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec785391-9afb-4393-9dc2-b0d614093020 admin admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:15.765175 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec785391-9afb-4393-9dc2-b0d614093020 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:15.767271 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec785391-9afb-4393-9dc2-b0d614093020 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:15.805133 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 913/1840] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:47:15 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:47:15.806167 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1d932f-1e64-4baa-b3eb-52296acf6604 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:15.829019 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1d932f-1e64-4baa-b3eb-52296acf6604 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:15.832382 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 928/1841] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:47:15 2022] POST /identity/v3/projects => generated 377 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:47:15.835726 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec785391-9afb-4393-9dc2-b0d614093020 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:15.857644 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-ec785391-9afb-4393-9dc2-b0d614093020 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:15.971528 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ac4e71-a5ed-4806-9ded-aa262dfea028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:15.972642 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ac4e71-a5ed-4806-9ded-aa262dfea028 admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:15.973087 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ac4e71-a5ed-4806-9ded-aa262dfea028 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:15.973394 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ac4e71-a5ed-4806-9ded-aa262dfea028 admin admin] PATH_INFO: `/v3/projects/ee12a5a4bf2441b6ac4ac7d5220b3dec/users/182869d74732426f8314d0eca33f7cde/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:15.982844 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec785391-9afb-4393-9dc2-b0d614093020 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:15.987963 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aee64d-9542-4a26-afde-99b63d6e2fef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:15.989328 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aee64d-9542-4a26-afde-99b63d6e2fef service cinder] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:15.989328 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aee64d-9542-4a26-afde-99b63d6e2fef service cinder] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:15.989507 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aee64d-9542-4a26-afde-99b63d6e2fef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:15.992078 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ac4e71-a5ed-4806-9ded-aa262dfea028 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee12a5a4bf2441b6ac4ac7d5220b3dec, user_id=182869d74732426f8314d0eca33f7cde, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:15.994820 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ac4e71-a5ed-4806-9ded-aa262dfea028 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:16.011236 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 914/1842] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:47:15 2022] PUT /identity/v3/projects/ee12a5a4bf2441b6ac4ac7d5220b3dec/users/182869d74732426f8314d0eca33f7cde/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:47:16.012182 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ac4e71-a5ed-4806-9ded-aa262dfea028 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:16.013761 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-19ac4e71-a5ed-4806-9ded-aa262dfea028 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:16.069268 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aee64d-9542-4a26-afde-99b63d6e2fef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:16.071059 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aee64d-9542-4a26-afde-99b63d6e2fef service cinder] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:16.081695 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 929/1843] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:47:15 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:47:16.082600 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aee64d-9542-4a26-afde-99b63d6e2fef service cinder] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:16.095560 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ac4e71-a5ed-4806-9ded-aa262dfea028 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:16.100657 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:16.102180 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34ccacb-cbcb-4ea9-9b76-0fad87905878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:16.104078 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34ccacb-cbcb-4ea9-9b76-0fad87905878 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:16.105524 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34ccacb-cbcb-4ea9-9b76-0fad87905878 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:16.105968 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34ccacb-cbcb-4ea9-9b76-0fad87905878 admin admin] PATH_INFO: `/v3/projects/600bf1fbb9134fcaaa38e2432c980d75` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:16.107723 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34ccacb-cbcb-4ea9-9b76-0fad87905878 admin admin] RBAC: Authorizing `identity:delete_project(project_id=600bf1fbb9134fcaaa38e2432c980d75)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:16.112333 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34ccacb-cbcb-4ea9-9b76-0fad87905878 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:16.113183 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aee64d-9542-4a26-afde-99b63d6e2fef service cinder] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:16.119283 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23706ade-7648-4736-9836-2e8e47fe6224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:16.120004 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23706ade-7648-4736-9836-2e8e47fe6224 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:16.120405 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23706ade-7648-4736-9836-2e8e47fe6224 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:16.120732 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23706ade-7648-4736-9836-2e8e47fe6224 admin admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:16.121470 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23706ade-7648-4736-9836-2e8e47fe6224 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:16.123631 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23706ade-7648-4736-9836-2e8e47fe6224 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:16.155577 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 930/1844] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:47:16 2022] POST /identity/v3/users => generated 289 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:47:16.155577 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23706ade-7648-4736-9836-2e8e47fe6224 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:16.157097 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-23706ade-7648-4736-9836-2e8e47fe6224 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:16.188019 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:16.189832 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-c34ccacb-cbcb-4ea9-9b76-0fad87905878 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '600bf1fbb9134fcaaa38e2432c980d75', 'request_id': 'req-c34ccacb-cbcb-4ea9-9b76-0fad87905878'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:16.199852 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-c34ccacb-cbcb-4ea9-9b76-0fad87905878 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '600bf1fbb9134fcaaa38e2432c980d75', 'request_id': 'req-c34ccacb-cbcb-4ea9-9b76-0fad87905878'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:16.210224 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-c34ccacb-cbcb-4ea9-9b76-0fad87905878 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '600bf1fbb9134fcaaa38e2432c980d75', 'request_id': 'req-c34ccacb-cbcb-4ea9-9b76-0fad87905878'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:16.225756 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-c34ccacb-cbcb-4ea9-9b76-0fad87905878 admin admin] The token cache is being invalidate because project 600bf1fbb9134fcaaa38e2432c980d75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90653) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:47:16.226049 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-c34ccacb-cbcb-4ea9-9b76-0fad87905878 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:16.226922 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-c34ccacb-cbcb-4ea9-9b76-0fad87905878 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:16.228863 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 915/1845] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:47:16 2022] DELETE /identity/v3/projects/600bf1fbb9134fcaaa38e2432c980d75 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:47:16.233904 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c6b23b-3e08-4bcc-8dc8-cc9eaa669699 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:16.233904 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c6b23b-3e08-4bcc-8dc8-cc9eaa669699 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:16.233904 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c6b23b-3e08-4bcc-8dc8-cc9eaa669699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:16.261396 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23706ade-7648-4736-9836-2e8e47fe6224 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:16.262264 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-15c6b23b-3e08-4bcc-8dc8-cc9eaa669699 None None] MFA Rules not processed for user `182869d74732426f8314d0eca33f7cde`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:47:16.263632 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-15c6b23b-3e08-4bcc-8dc8-cc9eaa669699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:16.342972 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 916/1846] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:47:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:47:16.343794 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c6b23b-3e08-4bcc-8dc8-cc9eaa669699 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:16.345206 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-15c6b23b-3e08-4bcc-8dc8-cc9eaa669699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:16.353120 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e1008b-a4cc-4b08-b22c-76e5f024dbba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bd5356829b6454993bab9bad2f79fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:16.353847 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1008b-a4cc-4b08-b22c-76e5f024dbba service nova] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:16.354189 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1008b-a4cc-4b08-b22c-76e5f024dbba service nova] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:16.354535 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1008b-a4cc-4b08-b22c-76e5f024dbba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:16.357187 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-f2e1008b-a4cc-4b08-b22c-76e5f024dbba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:16.370068 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e1008b-a4cc-4b08-b22c-76e5f024dbba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:16.371800 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e1008b-a4cc-4b08-b22c-76e5f024dbba service nova] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:16.384886 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 931/1847] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:47:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:47:16.386563 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e1008b-a4cc-4b08-b22c-76e5f024dbba service nova] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:16.388063 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-f2e1008b-a4cc-4b08-b22c-76e5f024dbba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:16.428773 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c6b23b-3e08-4bcc-8dc8-cc9eaa669699 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:16.436819 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2532b96e-869b-401d-9400-631b15af8a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:16.437516 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2532b96e-869b-401d-9400-631b15af8a19 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:16.437858 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2532b96e-869b-401d-9400-631b15af8a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:16.438635 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2532b96e-869b-401d-9400-631b15af8a19 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:16.440240 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2532b96e-869b-401d-9400-631b15af8a19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:16.442618 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2532b96e-869b-401d-9400-631b15af8a19 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:16.457956 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 917/1848] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:47:16 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:47:16.457956 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddb8fa6-163e-444d-a176-a72f641bf9be None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:16.457956 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddb8fa6-163e-444d-a176-a72f641bf9be None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:16.457956 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddb8fa6-163e-444d-a176-a72f641bf9be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:16.466179 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e1008b-a4cc-4b08-b22c-76e5f024dbba service nova] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:16.475984 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79868c33-4358-44f5-b7f5-afb580fb2c5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:16.476607 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79868c33-4358-44f5-b7f5-afb580fb2c5e service cinder] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:16.476893 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79868c33-4358-44f5-b7f5-afb580fb2c5e service cinder] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:16.477159 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79868c33-4358-44f5-b7f5-afb580fb2c5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:16.480177 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-79868c33-4358-44f5-b7f5-afb580fb2c5e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:16.491063 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-4ddb8fa6-163e-444d-a176-a72f641bf9be None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:47:16.492666 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-4ddb8fa6-163e-444d-a176-a72f641bf9be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:16.500696 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 918/1849] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:47:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:47:16.501592 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddb8fa6-163e-444d-a176-a72f641bf9be None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:16.504496 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-4ddb8fa6-163e-444d-a176-a72f641bf9be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:16.506080 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:16.517173 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddb8fa6-163e-444d-a176-a72f641bf9be None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:16.522784 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e9215b-230c-475c-aab0-7067ac92a32f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:16.523393 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e9215b-230c-475c-aab0-7067ac92a32f admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:16.523713 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e9215b-230c-475c-aab0-7067ac92a32f admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:16.524007 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e9215b-230c-475c-aab0-7067ac92a32f admin admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:16.524623 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e9215b-230c-475c-aab0-7067ac92a32f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:16.526536 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e9215b-230c-475c-aab0-7067ac92a32f admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:16.545525 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79868c33-4358-44f5-b7f5-afb580fb2c5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:16.547208 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79868c33-4358-44f5-b7f5-afb580fb2c5e service cinder] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:16.557596 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 919/1850] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:47:16 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:47:16.557596 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e9215b-230c-475c-aab0-7067ac92a32f admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:16.557862 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 932/1851] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:47:16 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:47:16.558492 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dea5b4-10fb-4701-8174-434afb4b24f6 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:16.558778 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dea5b4-10fb-4701-8174-434afb4b24f6 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:16.559049 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dea5b4-10fb-4701-8174-434afb4b24f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:16.582315 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-60dea5b4-10fb-4701-8174-434afb4b24f6 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:47:16.583532 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-60dea5b4-10fb-4701-8174-434afb4b24f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:16.599342 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:16.640065 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e9215b-230c-475c-aab0-7067ac92a32f admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:16.647472 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872b7143-c3a9-4e72-82e4-d87dd5e86e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:16.648236 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b7143-c3a9-4e72-82e4-d87dd5e86e69 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:16.648622 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b7143-c3a9-4e72-82e4-d87dd5e86e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:16.649453 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b7143-c3a9-4e72-82e4-d87dd5e86e69 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:16.649950 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872b7143-c3a9-4e72-82e4-d87dd5e86e69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:16.651486 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872b7143-c3a9-4e72-82e4-d87dd5e86e69 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:16.654848 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 933/1852] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:47:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:47:16.654848 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dea5b4-10fb-4701-8174-434afb4b24f6 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:16.666068 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 920/1853] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:47:16 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:47:16.666068 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872b7143-c3a9-4e72-82e4-d87dd5e86e69 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:16.666267 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dea5b4-10fb-4701-8174-434afb4b24f6 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:16.671854 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224eaf54-f532-4609-a4fc-9966605fde96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:16.672479 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224eaf54-f532-4609-a4fc-9966605fde96 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:16.672758 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224eaf54-f532-4609-a4fc-9966605fde96 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:16.673031 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224eaf54-f532-4609-a4fc-9966605fde96 admin admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:16.674080 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224eaf54-f532-4609-a4fc-9966605fde96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:16.676023 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872b7143-c3a9-4e72-82e4-d87dd5e86e69 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:16.677317 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224eaf54-f532-4609-a4fc-9966605fde96 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:16.680768 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25b4da9-c6b3-4984-a38a-80f205e47870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:16.681379 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25b4da9-c6b3-4984-a38a-80f205e47870 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:16.681935 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25b4da9-c6b3-4984-a38a-80f205e47870 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:16.681935 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25b4da9-c6b3-4984-a38a-80f205e47870 admin admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:16.682571 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25b4da9-c6b3-4984-a38a-80f205e47870 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:47:16.682869 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25b4da9-c6b3-4984-a38a-80f205e47870 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:16.684504 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25b4da9-c6b3-4984-a38a-80f205e47870 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:16.700467 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 921/1854] 10.222.0.15 () {58 vars in 1153 bytes} [Sun Sep 11 10:47:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:47:16.701003 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25b4da9-c6b3-4984-a38a-80f205e47870 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:16.702105 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 934/1855] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:47:16 2022] POST /identity/v3/users => generated 298 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:47:16.706379 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224eaf54-f532-4609-a4fc-9966605fde96 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:16.712733 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25b4da9-c6b3-4984-a38a-80f205e47870 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:16.716648 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224eaf54-f532-4609-a4fc-9966605fde96 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:16.718284 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee032869-9339-4917-bd38-239f26b23361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:16.719038 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee032869-9339-4917-bd38-239f26b23361 admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:16.719386 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee032869-9339-4917-bd38-239f26b23361 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:16.719752 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee032869-9339-4917-bd38-239f26b23361 admin admin] PATH_INFO: `/v3/projects/2882defcb26b4707934f2cff1f783f23/users/7af4c12ac0bd469aa6c2f1abba485f01/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:16.721837 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f6de73-59c8-40d6-8cbc-27cee70aaed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:16.722659 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f6de73-59c8-40d6-8cbc-27cee70aaed5 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:16.723010 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f6de73-59c8-40d6-8cbc-27cee70aaed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:16.723454 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f6de73-59c8-40d6-8cbc-27cee70aaed5 admin admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:16.724520 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f6de73-59c8-40d6-8cbc-27cee70aaed5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:16.726442 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f6de73-59c8-40d6-8cbc-27cee70aaed5 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:16.740745 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee032869-9339-4917-bd38-239f26b23361 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2882defcb26b4707934f2cff1f783f23, user_id=7af4c12ac0bd469aa6c2f1abba485f01, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:16.744844 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee032869-9339-4917-bd38-239f26b23361 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:16.759267 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 922/1856] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:47:16 2022] PUT /identity/v3/projects/2882defcb26b4707934f2cff1f783f23/users/7af4c12ac0bd469aa6c2f1abba485f01/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:47:16.760156 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee032869-9339-4917-bd38-239f26b23361 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:16.775008 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 935/1857] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:47:16 2022] POST /identity/v3/projects => generated 387 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:47:16.776330 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a968b-eb75-4442-b8f1-9e6b047f4060 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:16.776616 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a968b-eb75-4442-b8f1-9e6b047f4060 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:16.776892 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a968b-eb75-4442-b8f1-9e6b047f4060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:16.811661 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-f40a968b-eb75-4442-b8f1-9e6b047f4060 None None] MFA Rules not processed for user `7af4c12ac0bd469aa6c2f1abba485f01`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:47:16.813370 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-f40a968b-eb75-4442-b8f1-9e6b047f4060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:16.869950 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee032869-9339-4917-bd38-239f26b23361 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:16.951567 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 936/1858] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:47:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:47:16.953327 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40a968b-eb75-4442-b8f1-9e6b047f4060 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:16.966889 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328c2756-688d-4888-a5f7-3fc799c417ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:16.967633 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328c2756-688d-4888-a5f7-3fc799c417ac admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:16.967920 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328c2756-688d-4888-a5f7-3fc799c417ac admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:16.968191 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328c2756-688d-4888-a5f7-3fc799c417ac admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:16.968890 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328c2756-688d-4888-a5f7-3fc799c417ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:16.971721 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328c2756-688d-4888-a5f7-3fc799c417ac admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:16.983770 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 923/1859] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:47:16 2022] GET /identity/v3/roles => generated 1420 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:47:16.984613 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328c2756-688d-4888-a5f7-3fc799c417ac admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:16.997594 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328c2756-688d-4888-a5f7-3fc799c417ac admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:16.997782 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:17.003674 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77852bb-4240-471a-b9e5-a66ac2e09a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:17.004296 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77852bb-4240-471a-b9e5-a66ac2e09a78 admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:17.004586 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77852bb-4240-471a-b9e5-a66ac2e09a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:17.004950 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77852bb-4240-471a-b9e5-a66ac2e09a78 admin admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:17.005551 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77852bb-4240-471a-b9e5-a66ac2e09a78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:17.007575 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77852bb-4240-471a-b9e5-a66ac2e09a78 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:17.055289 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 924/1860] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:47:16 2022] POST /identity/v3/projects => generated 375 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:47:17.056411 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77852bb-4240-471a-b9e5-a66ac2e09a78 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:17.064686 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40a968b-eb75-4442-b8f1-9e6b047f4060 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:17.078455 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:17.146671 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe06fc6-e30e-40a3-a79e-bf46dcad14d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:17.148124 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe06fc6-e30e-40a3-a79e-bf46dcad14d1 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:17.148740 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe06fc6-e30e-40a3-a79e-bf46dcad14d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:17.149201 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe06fc6-e30e-40a3-a79e-bf46dcad14d1 admin admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:17.150989 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe06fc6-e30e-40a3-a79e-bf46dcad14d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:17.154192 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe06fc6-e30e-40a3-a79e-bf46dcad14d1 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:17.170564 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77852bb-4240-471a-b9e5-a66ac2e09a78 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:17.178663 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed8154a-e1a1-4aa9-9873-34b0f2f724a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:17.179714 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed8154a-e1a1-4aa9-9873-34b0f2f724a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:17.180893 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed8154a-e1a1-4aa9-9873-34b0f2f724a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:17.181485 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed8154a-e1a1-4aa9-9873-34b0f2f724a7 admin admin] PATH_INFO: `/v3/projects/5f5f114f69a04fa786bbd18344d8a15d/users/01e9fb97dd8548508745fc2c60fefee4/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:17.188510 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 937/1861] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:47:16 2022] POST /identity/v3/users => generated 294 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:47:17.190269 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe06fc6-e30e-40a3-a79e-bf46dcad14d1 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:17.206031 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed8154a-e1a1-4aa9-9873-34b0f2f724a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f5f114f69a04fa786bbd18344d8a15d, user_id=01e9fb97dd8548508745fc2c60fefee4, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:17.207882 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe06fc6-e30e-40a3-a79e-bf46dcad14d1 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:17.211207 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed8154a-e1a1-4aa9-9873-34b0f2f724a7 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:17.216274 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbc70b0-02c9-4435-a08f-32e33ce0ea28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:17.217298 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbc70b0-02c9-4435-a08f-32e33ce0ea28 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:17.217693 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbc70b0-02c9-4435-a08f-32e33ce0ea28 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:17.218782 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbc70b0-02c9-4435-a08f-32e33ce0ea28 admin admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:17.219915 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbc70b0-02c9-4435-a08f-32e33ce0ea28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:17.222868 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbc70b0-02c9-4435-a08f-32e33ce0ea28 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:17.235401 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 925/1862] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:47:17 2022] PUT /identity/v3/projects/5f5f114f69a04fa786bbd18344d8a15d/users/01e9fb97dd8548508745fc2c60fefee4/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:47:17.236822 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed8154a-e1a1-4aa9-9873-34b0f2f724a7 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:17.257136 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 938/1863] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:47:17 2022] POST /identity/v3/users => generated 294 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:47:17.258517 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbc70b0-02c9-4435-a08f-32e33ce0ea28 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:17.271777 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbc70b0-02c9-4435-a08f-32e33ce0ea28 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:17.333303 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed8154a-e1a1-4aa9-9873-34b0f2f724a7 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:17.341427 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18d61e3-c15d-4903-a001-9054ba93c695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:17.342124 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18d61e3-c15d-4903-a001-9054ba93c695 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:17.342411 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18d61e3-c15d-4903-a001-9054ba93c695 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:17.342691 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18d61e3-c15d-4903-a001-9054ba93c695 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:17.343665 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18d61e3-c15d-4903-a001-9054ba93c695 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:17.346619 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18d61e3-c15d-4903-a001-9054ba93c695 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:17.353927 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7210dd17-88b0-4d65-8afb-bf8c93b5c575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:17.354728 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7210dd17-88b0-4d65-8afb-bf8c93b5c575 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:17.355177 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7210dd17-88b0-4d65-8afb-bf8c93b5c575 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:17.355591 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7210dd17-88b0-4d65-8afb-bf8c93b5c575 admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:17.356379 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7210dd17-88b0-4d65-8afb-bf8c93b5c575 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:17.358653 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7210dd17-88b0-4d65-8afb-bf8c93b5c575 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:17.362026 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 926/1864] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:47:17 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:47:17.362998 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18d61e3-c15d-4903-a001-9054ba93c695 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:17.373732 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 939/1865] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:47:17 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:47:17.374834 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7210dd17-88b0-4d65-8afb-bf8c93b5c575 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:17.380319 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18d61e3-c15d-4903-a001-9054ba93c695 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:17.389277 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea05e679-8d8c-41cc-a67f-2d0fd001f13d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:17.389932 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7210dd17-88b0-4d65-8afb-bf8c93b5c575 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:17.390452 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea05e679-8d8c-41cc-a67f-2d0fd001f13d admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:17.391792 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea05e679-8d8c-41cc-a67f-2d0fd001f13d admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:17.392311 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea05e679-8d8c-41cc-a67f-2d0fd001f13d admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:17.393867 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea05e679-8d8c-41cc-a67f-2d0fd001f13d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:17.396460 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320d3c9c-770d-4686-8203-a2767b294b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:17.396623 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea05e679-8d8c-41cc-a67f-2d0fd001f13d admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:17.397269 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320d3c9c-770d-4686-8203-a2767b294b0a admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:17.397649 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320d3c9c-770d-4686-8203-a2767b294b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:17.398779 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320d3c9c-770d-4686-8203-a2767b294b0a admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:17.399609 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320d3c9c-770d-4686-8203-a2767b294b0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:17.401442 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320d3c9c-770d-4686-8203-a2767b294b0a admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:17.413752 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 940/1866] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:47:17 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:47:17.414972 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320d3c9c-770d-4686-8203-a2767b294b0a admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:17.415891 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 927/1867] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:47:17 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:47:17.417551 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea05e679-8d8c-41cc-a67f-2d0fd001f13d admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:17.426460 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320d3c9c-770d-4686-8203-a2767b294b0a admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:17.432086 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda746e-e386-4cf8-a9b1-e0d883994e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:17.433026 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda746e-e386-4cf8-a9b1-e0d883994e77 admin admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:17.433477 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda746e-e386-4cf8-a9b1-e0d883994e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:17.433885 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea05e679-8d8c-41cc-a67f-2d0fd001f13d admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:17.434003 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda746e-e386-4cf8-a9b1-e0d883994e77 admin admin] PATH_INFO: `/v3/projects/5f5f114f69a04fa786bbd18344d8a15d/users/01e9fb97dd8548508745fc2c60fefee4/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:17.436957 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edda746e-e386-4cf8-a9b1-e0d883994e77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f5f114f69a04fa786bbd18344d8a15d, user_id=01e9fb97dd8548508745fc2c60fefee4, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:17.440707 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d36d8c1-4ce0-44e5-9ee5-edb92289155c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:17.441185 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edda746e-e386-4cf8-a9b1-e0d883994e77 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:17.442186 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d36d8c1-4ce0-44e5-9ee5-edb92289155c admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:17.442833 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d36d8c1-4ce0-44e5-9ee5-edb92289155c admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:17.443289 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d36d8c1-4ce0-44e5-9ee5-edb92289155c admin admin] PATH_INFO: `/v3/projects/e880011e2fe846b284bedd029cc3d930/users/d79dac5532ff43309245a51caeca53fc/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:17.470321 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 941/1868] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:47:17 2022] PUT /identity/v3/projects/5f5f114f69a04fa786bbd18344d8a15d/users/01e9fb97dd8548508745fc2c60fefee4/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:47:17.480385 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda746e-e386-4cf8-a9b1-e0d883994e77 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:17.494137 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:17.494791 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d36d8c1-4ce0-44e5-9ee5-edb92289155c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e880011e2fe846b284bedd029cc3d930, user_id=d79dac5532ff43309245a51caeca53fc, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:17.501368 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d36d8c1-4ce0-44e5-9ee5-edb92289155c admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:17.545926 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 928/1869] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:47:17 2022] PUT /identity/v3/projects/e880011e2fe846b284bedd029cc3d930/users/d79dac5532ff43309245a51caeca53fc/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 10:47:17.550144 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62854cc5-9a70-4a17-8549-7831ccef42d7 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:17.550559 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62854cc5-9a70-4a17-8549-7831ccef42d7 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:17.550843 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62854cc5-9a70-4a17-8549-7831ccef42d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:17.599062 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-62854cc5-9a70-4a17-8549-7831ccef42d7 None None] MFA Rules not processed for user `01e9fb97dd8548508745fc2c60fefee4`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:47:17.600445 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-62854cc5-9a70-4a17-8549-7831ccef42d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:17.614103 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda746e-e386-4cf8-a9b1-e0d883994e77 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:17.678316 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:17.707537 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31faa549-defc-4424-a74b-4829e5c36350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:17.708903 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31faa549-defc-4424-a74b-4829e5c36350 admin admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:17.709382 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31faa549-defc-4424-a74b-4829e5c36350 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:17.709813 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31faa549-defc-4424-a74b-4829e5c36350 admin admin] PATH_INFO: `/v3/projects/2c1b35454b884624a9246994e2545d01/users/03b64db6b09442fa888388d13cdcdd2d/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:17.730966 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31faa549-defc-4424-a74b-4829e5c36350 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c1b35454b884624a9246994e2545d01, user_id=03b64db6b09442fa888388d13cdcdd2d, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:17.734528 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31faa549-defc-4424-a74b-4829e5c36350 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:17.735854 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 929/1870] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:47:17 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:47:17.738870 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db1058e-3661-4bd8-917d-49b9dee14e8b None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:17.738870 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db1058e-3661-4bd8-917d-49b9dee14e8b None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:17.739558 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db1058e-3661-4bd8-917d-49b9dee14e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:17.764222 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 942/1871] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:47:17 2022] PUT /identity/v3/projects/2c1b35454b884624a9246994e2545d01/users/03b64db6b09442fa888388d13cdcdd2d/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:47:17.765221 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31faa549-defc-4424-a74b-4829e5c36350 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:17.777636 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-9db1058e-3661-4bd8-917d-49b9dee14e8b None None] MFA Rules not processed for user `d79dac5532ff43309245a51caeca53fc`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:47:17.779331 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-9db1058e-3661-4bd8-917d-49b9dee14e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:17.845009 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31faa549-defc-4424-a74b-4829e5c36350 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:17.848863 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fea2b28-d79b-4cf6-9462-180b279c0f29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:17.849477 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fea2b28-d79b-4cf6-9462-180b279c0f29 service cinder] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:17.849760 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fea2b28-d79b-4cf6-9462-180b279c0f29 service cinder] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:17.850036 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fea2b28-d79b-4cf6-9462-180b279c0f29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:17.856053 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 930/1872] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:47:17 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:47:17.857244 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db1058e-3661-4bd8-917d-49b9dee14e8b None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:17.905209 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:17.916221 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fea2b28-d79b-4cf6-9462-180b279c0f29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:17.917753 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fea2b28-d79b-4cf6-9462-180b279c0f29 service cinder] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:17.927265 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 943/1873] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:47:17 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:47:17.927885 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db1058e-3661-4bd8-917d-49b9dee14e8b None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:17.928106 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fea2b28-d79b-4cf6-9462-180b279c0f29 service cinder] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:17.933734 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f0a8a9-a8a5-4d82-bb74-052ed39c360c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:17.935081 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f0a8a9-a8a5-4d82-bb74-052ed39c360c admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:17.935469 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f0a8a9-a8a5-4d82-bb74-052ed39c360c admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:17.935731 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f0a8a9-a8a5-4d82-bb74-052ed39c360c admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:17.936680 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f0a8a9-a8a5-4d82-bb74-052ed39c360c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:17.938576 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f0a8a9-a8a5-4d82-bb74-052ed39c360c admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:17.938685 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fea2b28-d79b-4cf6-9462-180b279c0f29 service cinder] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:17.943572 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5572c0cc-4b6f-4791-9e4f-3e36aac59b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:17.944261 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5572c0cc-4b6f-4791-9e4f-3e36aac59b49 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:17.944627 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5572c0cc-4b6f-4791-9e4f-3e36aac59b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:17.944969 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5572c0cc-4b6f-4791-9e4f-3e36aac59b49 admin admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:17.945680 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5572c0cc-4b6f-4791-9e4f-3e36aac59b49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:17.947953 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5572c0cc-4b6f-4791-9e4f-3e36aac59b49 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:17.950413 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 931/1874] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:47:17 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:47:17.956038 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f0a8a9-a8a5-4d82-bb74-052ed39c360c admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:17.968432 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f0a8a9-a8a5-4d82-bb74-052ed39c360c admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:17.975998 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14779ad1-ceab-4564-89d2-de1b37b7673d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:17.976751 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14779ad1-ceab-4564-89d2-de1b37b7673d admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:17.977110 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14779ad1-ceab-4564-89d2-de1b37b7673d admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:17.977345 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14779ad1-ceab-4564-89d2-de1b37b7673d admin admin] PATH_INFO: `/v3/projects/2c1b35454b884624a9246994e2545d01/users/03b64db6b09442fa888388d13cdcdd2d/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:17.979837 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14779ad1-ceab-4564-89d2-de1b37b7673d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c1b35454b884624a9246994e2545d01, user_id=03b64db6b09442fa888388d13cdcdd2d, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:17.984022 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14779ad1-ceab-4564-89d2-de1b37b7673d admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:17.988920 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 944/1875] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:47:17 2022] POST /identity/v3/projects => generated 387 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:47:17.993312 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5572c0cc-4b6f-4791-9e4f-3e36aac59b49 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:18.000019 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 932/1876] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:47:17 2022] PUT /identity/v3/projects/2c1b35454b884624a9246994e2545d01/users/03b64db6b09442fa888388d13cdcdd2d/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 10:47:18.011191 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94716cd2-a10b-49b1-ba07-b3de01cb7d75 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:18.011511 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94716cd2-a10b-49b1-ba07-b3de01cb7d75 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:18.011782 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94716cd2-a10b-49b1-ba07-b3de01cb7d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:18.043968 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-94716cd2-a10b-49b1-ba07-b3de01cb7d75 None None] MFA Rules not processed for user `03b64db6b09442fa888388d13cdcdd2d`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:47:18.045258 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-94716cd2-a10b-49b1-ba07-b3de01cb7d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:18.081727 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5572c0cc-4b6f-4791-9e4f-3e36aac59b49 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:18.096502 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:18.137931 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 933/1877] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:47:18 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:47:18.160119 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c63dbfd-63bb-48e6-9941-136be0ed48e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:18.161400 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c63dbfd-63bb-48e6-9941-136be0ed48e2 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:18.161710 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c63dbfd-63bb-48e6-9941-136be0ed48e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:18.161991 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c63dbfd-63bb-48e6-9941-136be0ed48e2 admin admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:18.162692 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c63dbfd-63bb-48e6-9941-136be0ed48e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:18.164746 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c63dbfd-63bb-48e6-9941-136be0ed48e2 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:18.191007 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 945/1878] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:47:17 2022] POST /identity/v3/users => generated 300 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 10:47:18.192294 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94716cd2-a10b-49b1-ba07-b3de01cb7d75 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:18.203874 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94716cd2-a10b-49b1-ba07-b3de01cb7d75 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:18.208924 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb06267-1cc1-43d2-97de-98462ca7767d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:18.209546 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb06267-1cc1-43d2-97de-98462ca7767d admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:18.209869 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb06267-1cc1-43d2-97de-98462ca7767d admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:18.210137 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb06267-1cc1-43d2-97de-98462ca7767d admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:18.210782 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb06267-1cc1-43d2-97de-98462ca7767d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:18.212522 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb06267-1cc1-43d2-97de-98462ca7767d admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:18.223945 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 934/1879] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:47:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:47:18.227984 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c63dbfd-63bb-48e6-9941-136be0ed48e2 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:18.243225 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c63dbfd-63bb-48e6-9941-136be0ed48e2 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:18.248864 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bc7ed7-b620-4066-99ca-21aeffdbdb17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:18.249533 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc7ed7-b620-4066-99ca-21aeffdbdb17 admin admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:18.249814 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc7ed7-b620-4066-99ca-21aeffdbdb17 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:18.250089 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc7ed7-b620-4066-99ca-21aeffdbdb17 admin admin] PATH_INFO: `/v3/projects/d2a895010a9f459a84f37be46b99be61/users/4b09e38ebe744262bd52a9340a6012e2/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:18.274351 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb06267-1cc1-43d2-97de-98462ca7767d admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:18.290243 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bc7ed7-b620-4066-99ca-21aeffdbdb17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2a895010a9f459a84f37be46b99be61, user_id=4b09e38ebe744262bd52a9340a6012e2, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:18.290243 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bc7ed7-b620-4066-99ca-21aeffdbdb17 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:18.290420 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-1bb06267-1cc1-43d2-97de-98462ca7767d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:18.308535 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 946/1880] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:47:18 2022] PUT /identity/v3/projects/d2a895010a9f459a84f37be46b99be61/users/4b09e38ebe744262bd52a9340a6012e2/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 10:47:18.312431 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bc7ed7-b620-4066-99ca-21aeffdbdb17 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:18.397338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb06267-1cc1-43d2-97de-98462ca7767d admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:18.401624 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:18.452125 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bc7ed7-b620-4066-99ca-21aeffdbdb17 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:18.460809 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727d5ecd-16b2-4b1a-93ed-c31095869771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:18.462849 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d5ecd-16b2-4b1a-93ed-c31095869771 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:18.463366 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d5ecd-16b2-4b1a-93ed-c31095869771 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:18.463816 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d5ecd-16b2-4b1a-93ed-c31095869771 admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:18.465191 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727d5ecd-16b2-4b1a-93ed-c31095869771 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:18.467812 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727d5ecd-16b2-4b1a-93ed-c31095869771 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:18.479849 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4246bcb2-2e35-44bb-94d8-0d79b55c7e45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:18.481535 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4246bcb2-2e35-44bb-94d8-0d79b55c7e45 service cinder] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:18.482320 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4246bcb2-2e35-44bb-94d8-0d79b55c7e45 service cinder] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:18.482910 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4246bcb2-2e35-44bb-94d8-0d79b55c7e45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:18.485495 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 947/1881] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:47:18 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:47:18.486911 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727d5ecd-16b2-4b1a-93ed-c31095869771 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:18.500470 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727d5ecd-16b2-4b1a-93ed-c31095869771 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:18.507352 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710ad659-0bd0-43b1-96e8-723f5214a0ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:18.508065 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710ad659-0bd0-43b1-96e8-723f5214a0ae admin admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:18.508346 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710ad659-0bd0-43b1-96e8-723f5214a0ae admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:18.508625 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710ad659-0bd0-43b1-96e8-723f5214a0ae admin admin] PATH_INFO: `/v3/projects/d2a895010a9f459a84f37be46b99be61/users/4b09e38ebe744262bd52a9340a6012e2/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:18.512254 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710ad659-0bd0-43b1-96e8-723f5214a0ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2a895010a9f459a84f37be46b99be61, user_id=4b09e38ebe744262bd52a9340a6012e2, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:18.516049 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710ad659-0bd0-43b1-96e8-723f5214a0ae admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:18.527978 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 948/1882] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:47:18 2022] PUT /identity/v3/projects/d2a895010a9f459a84f37be46b99be61/users/4b09e38ebe744262bd52a9340a6012e2/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:47:18.539055 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc89150-8bc5-463f-bf4a-b4fd1753d399 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:18.539472 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc89150-8bc5-463f-bf4a-b4fd1753d399 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:18.539883 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc89150-8bc5-463f-bf4a-b4fd1753d399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:18.578328 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-9bc89150-8bc5-463f-bf4a-b4fd1753d399 None None] MFA Rules not processed for user `4b09e38ebe744262bd52a9340a6012e2`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:47:18.580017 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-9bc89150-8bc5-463f-bf4a-b4fd1753d399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:18.592050 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4246bcb2-2e35-44bb-94d8-0d79b55c7e45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:18.593492 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4246bcb2-2e35-44bb-94d8-0d79b55c7e45 service cinder] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:18.611698 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 935/1883] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:47:18 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:47:18.673371 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 949/1884] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:47:18 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:47:18.684868 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4246bcb2-2e35-44bb-94d8-0d79b55c7e45 service cinder] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:18.694813 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4246bcb2-2e35-44bb-94d8-0d79b55c7e45 service cinder] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:18.734672 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:18.749096 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0033d7-b7d7-4ffc-8f65-a09ae8cd3521 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:18.749915 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0033d7-b7d7-4ffc-8f65-a09ae8cd3521 service cinder] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:18.750404 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0033d7-b7d7-4ffc-8f65-a09ae8cd3521 service cinder] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:18.750761 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0033d7-b7d7-4ffc-8f65-a09ae8cd3521 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:18.761990 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0033d7-b7d7-4ffc-8f65-a09ae8cd3521 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:18.763451 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0033d7-b7d7-4ffc-8f65-a09ae8cd3521 service cinder] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:18.775040 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 936/1885] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:47:18 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:47:18.804911 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc89150-8bc5-463f-bf4a-b4fd1753d399 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:18.904996 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc89150-8bc5-463f-bf4a-b4fd1753d399 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:18.916017 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ec320a-e23d-4081-b050-58824653c8fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:18.916951 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ec320a-e23d-4081-b050-58824653c8fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:18.919729 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ec320a-e23d-4081-b050-58824653c8fd admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:18.920524 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ec320a-e23d-4081-b050-58824653c8fd admin admin] PATH_INFO: `/v3/users/d79dac5532ff43309245a51caeca53fc` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:18.922203 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ec320a-e23d-4081-b050-58824653c8fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=d79dac5532ff43309245a51caeca53fc)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:18.926707 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ec320a-e23d-4081-b050-58824653c8fd admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:18.999417 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-c6ec320a-e23d-4081-b050-58824653c8fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd79dac5532ff43309245a51caeca53fc'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:19.002158 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:19.011995 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-c6ec320a-e23d-4081-b050-58824653c8fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd79dac5532ff43309245a51caeca53fc'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:19.012933 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-c6ec320a-e23d-4081-b050-58824653c8fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd79dac5532ff43309245a51caeca53fc'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:19.013632 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-c6ec320a-e23d-4081-b050-58824653c8fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd79dac5532ff43309245a51caeca53fc'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:19.034977 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-c6ec320a-e23d-4081-b050-58824653c8fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd79dac5532ff43309245a51caeca53fc'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:19.047235 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 950/1886] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:47:18 2022] DELETE /identity/v3/users/d79dac5532ff43309245a51caeca53fc => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:47:19.052179 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0189a36-8bb1-47cf-88be-793f7fe6fda6 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:19.052470 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0189a36-8bb1-47cf-88be-793f7fe6fda6 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:19.052747 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0189a36-8bb1-47cf-88be-793f7fe6fda6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:19.072166 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-a0189a36-8bb1-47cf-88be-793f7fe6fda6 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:47:19.073363 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-a0189a36-8bb1-47cf-88be-793f7fe6fda6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:19.132595 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 937/1887] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:47:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:47:19.142436 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ec320a-e23d-4081-b050-58824653c8fd admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:19.144291 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-c6ec320a-e23d-4081-b050-58824653c8fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:19.201512 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ec320a-e23d-4081-b050-58824653c8fd admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:19.205013 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170167c2-961d-4586-bb4c-c5ab518dbf66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:19.205736 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170167c2-961d-4586-bb4c-c5ab518dbf66 service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:19.206015 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170167c2-961d-4586-bb4c-c5ab518dbf66 service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:19.206287 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170167c2-961d-4586-bb4c-c5ab518dbf66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:19.215223 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170167c2-961d-4586-bb4c-c5ab518dbf66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:19.216674 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170167c2-961d-4586-bb4c-c5ab518dbf66 service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:19.226147 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 951/1888] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:47:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:47:19.507591 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0189a36-8bb1-47cf-88be-793f7fe6fda6 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:19.509260 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-a0189a36-8bb1-47cf-88be-793f7fe6fda6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:19.541328 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0189a36-8bb1-47cf-88be-793f7fe6fda6 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:19.546410 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3956826-f3f6-423e-af3d-1e46f6cb9097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:19.547114 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3956826-f3f6-423e-af3d-1e46f6cb9097 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:19.547552 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3956826-f3f6-423e-af3d-1e46f6cb9097 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:19.547898 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3956826-f3f6-423e-af3d-1e46f6cb9097 admin admin] PATH_INFO: `/v3/projects/e880011e2fe846b284bedd029cc3d930` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:19.549991 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3956826-f3f6-423e-af3d-1e46f6cb9097 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e880011e2fe846b284bedd029cc3d930)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:19.560064 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3956826-f3f6-423e-af3d-1e46f6cb9097 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:19.648557 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-e3956826-f3f6-423e-af3d-1e46f6cb9097 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e880011e2fe846b284bedd029cc3d930', 'request_id': 'req-e3956826-f3f6-423e-af3d-1e46f6cb9097'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:19.659950 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-e3956826-f3f6-423e-af3d-1e46f6cb9097 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e880011e2fe846b284bedd029cc3d930', 'request_id': 'req-e3956826-f3f6-423e-af3d-1e46f6cb9097'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:19.672002 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-e3956826-f3f6-423e-af3d-1e46f6cb9097 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e880011e2fe846b284bedd029cc3d930', 'request_id': 'req-e3956826-f3f6-423e-af3d-1e46f6cb9097'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:19.700635 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-e3956826-f3f6-423e-af3d-1e46f6cb9097 admin admin] The token cache is being invalidate because project e880011e2fe846b284bedd029cc3d930 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90653) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:47:19.700931 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-e3956826-f3f6-423e-af3d-1e46f6cb9097 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:19.701870 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-e3956826-f3f6-423e-af3d-1e46f6cb9097 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:19.703914 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 938/1889] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:47:19 2022] DELETE /identity/v3/projects/e880011e2fe846b284bedd029cc3d930 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:47:19.707936 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170167c2-961d-4586-bb4c-c5ab518dbf66 service neutron] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:19.722930 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170167c2-961d-4586-bb4c-c5ab518dbf66 service neutron] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:19.821825 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5045f2b5-c7c0-4138-b78e-3691286e91a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:19.822471 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5045f2b5-c7c0-4138-b78e-3691286e91a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:19.822808 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5045f2b5-c7c0-4138-b78e-3691286e91a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:19.823539 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5045f2b5-c7c0-4138-b78e-3691286e91a3 admin admin] PATH_INFO: `/v3/users/4b09e38ebe744262bd52a9340a6012e2` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:19.824813 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5045f2b5-c7c0-4138-b78e-3691286e91a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b09e38ebe744262bd52a9340a6012e2)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:19.826938 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5045f2b5-c7c0-4138-b78e-3691286e91a3 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:19.917243 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-5045f2b5-c7c0-4138-b78e-3691286e91a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b09e38ebe744262bd52a9340a6012e2'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:19.947558 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-5045f2b5-c7c0-4138-b78e-3691286e91a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b09e38ebe744262bd52a9340a6012e2'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:19.948730 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-5045f2b5-c7c0-4138-b78e-3691286e91a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b09e38ebe744262bd52a9340a6012e2'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:19.949578 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-5045f2b5-c7c0-4138-b78e-3691286e91a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b09e38ebe744262bd52a9340a6012e2'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:19.983028 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-5045f2b5-c7c0-4138-b78e-3691286e91a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b09e38ebe744262bd52a9340a6012e2'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:19.987692 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:20.019947 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 952/1890] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:47:19 2022] DELETE /identity/v3/users/4b09e38ebe744262bd52a9340a6012e2 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:47:20.567501 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3956826-f3f6-423e-af3d-1e46f6cb9097 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:20.568997 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-e3956826-f3f6-423e-af3d-1e46f6cb9097 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:20.582947 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3956826-f3f6-423e-af3d-1e46f6cb9097 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:20.647078 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfd704a-e79b-4aec-91f6-33e6037ac201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:20.647756 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd704a-e79b-4aec-91f6-33e6037ac201 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:20.648042 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd704a-e79b-4aec-91f6-33e6037ac201 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:20.648319 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd704a-e79b-4aec-91f6-33e6037ac201 admin admin] PATH_INFO: `/v3/projects/d2a895010a9f459a84f37be46b99be61` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:20.649449 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfd704a-e79b-4aec-91f6-33e6037ac201 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2a895010a9f459a84f37be46b99be61)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:20.651591 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfd704a-e79b-4aec-91f6-33e6037ac201 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:20.663763 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:20.706974 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-ebfd704a-e79b-4aec-91f6-33e6037ac201 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2a895010a9f459a84f37be46b99be61', 'request_id': 'req-ebfd704a-e79b-4aec-91f6-33e6037ac201'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:20.713316 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-ebfd704a-e79b-4aec-91f6-33e6037ac201 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2a895010a9f459a84f37be46b99be61', 'request_id': 'req-ebfd704a-e79b-4aec-91f6-33e6037ac201'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:20.720591 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-ebfd704a-e79b-4aec-91f6-33e6037ac201 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2a895010a9f459a84f37be46b99be61', 'request_id': 'req-ebfd704a-e79b-4aec-91f6-33e6037ac201'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:20.731410 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-ebfd704a-e79b-4aec-91f6-33e6037ac201 admin admin] The token cache is being invalidate because project d2a895010a9f459a84f37be46b99be61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90653) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:47:20.731992 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-ebfd704a-e79b-4aec-91f6-33e6037ac201 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:20.732635 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-ebfd704a-e79b-4aec-91f6-33e6037ac201 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:20.734231 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 939/1891] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:47:20 2022] DELETE /identity/v3/projects/d2a895010a9f459a84f37be46b99be61 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 10:47:20.740341 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d40bf-130f-4ce2-886d-f6afb0838980 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:20.740855 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d40bf-130f-4ce2-886d-f6afb0838980 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:20.740975 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d40bf-130f-4ce2-886d-f6afb0838980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:20.760958 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-884d40bf-130f-4ce2-886d-f6afb0838980 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:47:20.762161 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-884d40bf-130f-4ce2-886d-f6afb0838980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:20.825491 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 953/1892] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:47:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:47:20.833262 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80258b8d-a6fb-497a-9a4f-0c23532129dd None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:20.833553 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80258b8d-a6fb-497a-9a4f-0c23532129dd None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:20.833867 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80258b8d-a6fb-497a-9a4f-0c23532129dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:20.855131 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-80258b8d-a6fb-497a-9a4f-0c23532129dd None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:47:20.856396 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-80258b8d-a6fb-497a-9a4f-0c23532129dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:20.865730 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 940/1893] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:47:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:47:20.869966 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d40bf-130f-4ce2-886d-f6afb0838980 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:20.880715 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d40bf-130f-4ce2-886d-f6afb0838980 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:20.885590 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2773934-73f5-4570-b1e4-aed166b22584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:20.886307 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2773934-73f5-4570-b1e4-aed166b22584 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:20.886672 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2773934-73f5-4570-b1e4-aed166b22584 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:20.886950 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2773934-73f5-4570-b1e4-aed166b22584 admin admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:20.887652 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2773934-73f5-4570-b1e4-aed166b22584 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:47:20.887955 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2773934-73f5-4570-b1e4-aed166b22584 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:20.889584 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2773934-73f5-4570-b1e4-aed166b22584 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:20.905115 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 954/1894] 10.222.0.15 () {58 vars in 1153 bytes} [Sun Sep 11 10:47:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:47:20.906335 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80258b8d-a6fb-497a-9a4f-0c23532129dd None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:20.916750 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80258b8d-a6fb-497a-9a4f-0c23532129dd None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:20.921359 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e08b560-743a-4d19-8433-3153865cc2ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:20.925343 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e08b560-743a-4d19-8433-3153865cc2ec admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:20.925680 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e08b560-743a-4d19-8433-3153865cc2ec admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:20.925963 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e08b560-743a-4d19-8433-3153865cc2ec admin admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:20.926649 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e08b560-743a-4d19-8433-3153865cc2ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:20.928823 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e08b560-743a-4d19-8433-3153865cc2ec admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:20.956591 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 941/1895] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:47:20 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:47:20.960613 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2773934-73f5-4570-b1e4-aed166b22584 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:20.971214 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2773934-73f5-4570-b1e4-aed166b22584 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:21.037739 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d0044c-a314-44ea-8e83-b5b179c84e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:21.038351 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d0044c-a314-44ea-8e83-b5b179c84e3c admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:21.038630 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d0044c-a314-44ea-8e83-b5b179c84e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:21.038902 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d0044c-a314-44ea-8e83-b5b179c84e3c admin admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:21.039552 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d0044c-a314-44ea-8e83-b5b179c84e3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:21.041448 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d0044c-a314-44ea-8e83-b5b179c84e3c admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:21.064723 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 955/1896] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:47:20 2022] POST /identity/v3/users => generated 286 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 10:47:21.069095 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e08b560-743a-4d19-8433-3153865cc2ec admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:21.079910 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e08b560-743a-4d19-8433-3153865cc2ec admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:21.085607 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77d58a8-886e-4695-9cc6-fa1cef794ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:21.086434 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d58a8-886e-4695-9cc6-fa1cef794ac0 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:21.086682 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d58a8-886e-4695-9cc6-fa1cef794ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:21.087224 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d58a8-886e-4695-9cc6-fa1cef794ac0 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:21.088442 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77d58a8-886e-4695-9cc6-fa1cef794ac0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:21.090222 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77d58a8-886e-4695-9cc6-fa1cef794ac0 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:21.102616 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 942/1897] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:47:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:47:21.106430 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d0044c-a314-44ea-8e83-b5b179c84e3c admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:21.117381 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d0044c-a314-44ea-8e83-b5b179c84e3c admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:21.122417 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfd5d59-ffa0-4837-a0c1-58db5e77c399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:21.123118 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfd5d59-ffa0-4837-a0c1-58db5e77c399 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:21.123376 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfd5d59-ffa0-4837-a0c1-58db5e77c399 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:21.123689 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfd5d59-ffa0-4837-a0c1-58db5e77c399 admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:21.124483 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfd5d59-ffa0-4837-a0c1-58db5e77c399 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:21.126249 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfd5d59-ffa0-4837-a0c1-58db5e77c399 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:21.138870 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 956/1898] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:47:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:47:21.143266 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77d58a8-886e-4695-9cc6-fa1cef794ac0 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:21.153536 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77d58a8-886e-4695-9cc6-fa1cef794ac0 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:21.159005 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61abce2-907a-455a-b517-f103ba61862a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:21.159923 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61abce2-907a-455a-b517-f103ba61862a admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:21.160320 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61abce2-907a-455a-b517-f103ba61862a admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:21.160591 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61abce2-907a-455a-b517-f103ba61862a admin admin] PATH_INFO: `/v3/projects/ff35e66169d4416cbc7f79e53bc18c10/users/bdab170413984e39bf2993025ae944fe/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:21.176057 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61abce2-907a-455a-b517-f103ba61862a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff35e66169d4416cbc7f79e53bc18c10, user_id=bdab170413984e39bf2993025ae944fe, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:21.178839 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61abce2-907a-455a-b517-f103ba61862a admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:21.193445 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 943/1899] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:47:21 2022] PUT /identity/v3/projects/ff35e66169d4416cbc7f79e53bc18c10/users/bdab170413984e39bf2993025ae944fe/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 10:47:21.204919 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2a1398-8389-4a83-8578-cbe1f3d8ccc0 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:21.205204 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2a1398-8389-4a83-8578-cbe1f3d8ccc0 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:21.205635 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2a1398-8389-4a83-8578-cbe1f3d8ccc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:21.235640 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-0c2a1398-8389-4a83-8578-cbe1f3d8ccc0 None None] MFA Rules not processed for user `bdab170413984e39bf2993025ae944fe`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:47:21.238080 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-0c2a1398-8389-4a83-8578-cbe1f3d8ccc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:21.287663 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:21.315959 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 957/1900] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:47:21 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:47:21.327210 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61abce2-907a-455a-b517-f103ba61862a admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:21.334129 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-f61abce2-907a-455a-b517-f103ba61862a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:21.399569 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61abce2-907a-455a-b517-f103ba61862a admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:21.405255 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea24e75-f647-4cba-8e8d-e979a832a8d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:21.406102 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea24e75-f647-4cba-8e8d-e979a832a8d1 service cinder] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:21.406439 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea24e75-f647-4cba-8e8d-e979a832a8d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:21.406651 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea24e75-f647-4cba-8e8d-e979a832a8d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:21.417687 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea24e75-f647-4cba-8e8d-e979a832a8d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:21.419544 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea24e75-f647-4cba-8e8d-e979a832a8d1 service cinder] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:21.433699 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 944/1901] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:47:21 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:47:21.457080 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2a1398-8389-4a83-8578-cbe1f3d8ccc0 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:21.530306 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2a1398-8389-4a83-8578-cbe1f3d8ccc0 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:21.535780 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac17d3c0-bb70-45be-8144-de19ced4fc4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:21.536416 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac17d3c0-bb70-45be-8144-de19ced4fc4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:21.536693 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac17d3c0-bb70-45be-8144-de19ced4fc4c admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:21.536960 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac17d3c0-bb70-45be-8144-de19ced4fc4c admin admin] PATH_INFO: `/v3/users/bdab170413984e39bf2993025ae944fe` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:21.538081 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac17d3c0-bb70-45be-8144-de19ced4fc4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdab170413984e39bf2993025ae944fe)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:21.540059 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac17d3c0-bb70-45be-8144-de19ced4fc4c admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:21.589318 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-ac17d3c0-bb70-45be-8144-de19ced4fc4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdab170413984e39bf2993025ae944fe'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:21.598206 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-ac17d3c0-bb70-45be-8144-de19ced4fc4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdab170413984e39bf2993025ae944fe'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:21.598828 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-ac17d3c0-bb70-45be-8144-de19ced4fc4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdab170413984e39bf2993025ae944fe'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:21.599564 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-ac17d3c0-bb70-45be-8144-de19ced4fc4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdab170413984e39bf2993025ae944fe'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:21.613911 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-ac17d3c0-bb70-45be-8144-de19ced4fc4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdab170413984e39bf2993025ae944fe'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:21.635579 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 958/1902] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:47:21 2022] DELETE /identity/v3/users/bdab170413984e39bf2993025ae944fe => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:47:21.637526 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced4ae4a-5a25-48c3-9162-7ba389662cd2 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:21.637836 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced4ae4a-5a25-48c3-9162-7ba389662cd2 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:21.638089 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced4ae4a-5a25-48c3-9162-7ba389662cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:21.660456 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-ced4ae4a-5a25-48c3-9162-7ba389662cd2 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:47:21.661684 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-ced4ae4a-5a25-48c3-9162-7ba389662cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:21.692431 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:21.723756 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 945/1903] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:47:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:47:21.733862 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac17d3c0-bb70-45be-8144-de19ced4fc4c admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:21.735590 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-ac17d3c0-bb70-45be-8144-de19ced4fc4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:21.795671 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac17d3c0-bb70-45be-8144-de19ced4fc4c admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:21.799264 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a6c0a9-3615-4076-9383-c4970be88696 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:21.799883 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a6c0a9-3615-4076-9383-c4970be88696 service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:21.800164 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a6c0a9-3615-4076-9383-c4970be88696 service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:21.800437 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a6c0a9-3615-4076-9383-c4970be88696 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:21.809217 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a6c0a9-3615-4076-9383-c4970be88696 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:21.810632 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a6c0a9-3615-4076-9383-c4970be88696 service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:21.820372 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 959/1904] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:47:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:47:22.093490 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced4ae4a-5a25-48c3-9162-7ba389662cd2 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:22.094979 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-ced4ae4a-5a25-48c3-9162-7ba389662cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:22.108213 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced4ae4a-5a25-48c3-9162-7ba389662cd2 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:22.116746 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649fa603-b854-4a81-9680-1e036a37a8a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:22.117508 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649fa603-b854-4a81-9680-1e036a37a8a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:22.118268 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649fa603-b854-4a81-9680-1e036a37a8a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:22.118552 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649fa603-b854-4a81-9680-1e036a37a8a4 admin admin] PATH_INFO: `/v3/projects/ff35e66169d4416cbc7f79e53bc18c10` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:22.119763 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649fa603-b854-4a81-9680-1e036a37a8a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff35e66169d4416cbc7f79e53bc18c10)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:22.121682 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649fa603-b854-4a81-9680-1e036a37a8a4 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:22.171391 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-649fa603-b854-4a81-9680-1e036a37a8a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff35e66169d4416cbc7f79e53bc18c10', 'request_id': 'req-649fa603-b854-4a81-9680-1e036a37a8a4'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:22.178307 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-649fa603-b854-4a81-9680-1e036a37a8a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff35e66169d4416cbc7f79e53bc18c10', 'request_id': 'req-649fa603-b854-4a81-9680-1e036a37a8a4'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:22.186341 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-649fa603-b854-4a81-9680-1e036a37a8a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff35e66169d4416cbc7f79e53bc18c10', 'request_id': 'req-649fa603-b854-4a81-9680-1e036a37a8a4'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:22.199005 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-649fa603-b854-4a81-9680-1e036a37a8a4 admin admin] The token cache is being invalidate because project ff35e66169d4416cbc7f79e53bc18c10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90653) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:47:22.199299 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-649fa603-b854-4a81-9680-1e036a37a8a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:22.199978 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-649fa603-b854-4a81-9680-1e036a37a8a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:22.201587 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 946/1905] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:47:22 2022] DELETE /identity/v3/projects/ff35e66169d4416cbc7f79e53bc18c10 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:47:22.206717 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7db391-cd24-4613-9e8d-6aba2fbd3181 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:22.207008 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7db391-cd24-4613-9e8d-6aba2fbd3181 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:22.207286 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7db391-cd24-4613-9e8d-6aba2fbd3181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:22.220965 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:22.229126 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-6f7db391-cd24-4613-9e8d-6aba2fbd3181 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:47:22.230818 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-6f7db391-cd24-4613-9e8d-6aba2fbd3181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:22.292839 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 960/1906] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:47:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:47:22.301261 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f2e7dc-b857-4dfb-8f56-3c0958a1b4bd None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:22.301549 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f2e7dc-b857-4dfb-8f56-3c0958a1b4bd None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:22.301822 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f2e7dc-b857-4dfb-8f56-3c0958a1b4bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:22.324551 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-b8f2e7dc-b857-4dfb-8f56-3c0958a1b4bd None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:47:22.325758 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-b8f2e7dc-b857-4dfb-8f56-3c0958a1b4bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:22.333654 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 947/1907] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:47:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:47:22.337848 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7db391-cd24-4613-9e8d-6aba2fbd3181 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:22.347479 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7db391-cd24-4613-9e8d-6aba2fbd3181 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:22.351984 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b55de9-4ac0-47e7-b2be-058826d773c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:22.352572 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b55de9-4ac0-47e7-b2be-058826d773c7 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:22.352849 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b55de9-4ac0-47e7-b2be-058826d773c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:22.353235 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b55de9-4ac0-47e7-b2be-058826d773c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:22.353875 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b55de9-4ac0-47e7-b2be-058826d773c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:47:22.354170 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b55de9-4ac0-47e7-b2be-058826d773c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:22.355776 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b55de9-4ac0-47e7-b2be-058826d773c7 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:22.368981 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 961/1908] 10.222.0.15 () {58 vars in 1153 bytes} [Sun Sep 11 10:47:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:47:22.372919 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f2e7dc-b857-4dfb-8f56-3c0958a1b4bd None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:22.382695 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f2e7dc-b857-4dfb-8f56-3c0958a1b4bd None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:22.387287 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0581dd48-bf06-4f22-873a-41d96e3c5e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:22.387918 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0581dd48-bf06-4f22-873a-41d96e3c5e3a admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:22.388200 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0581dd48-bf06-4f22-873a-41d96e3c5e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:22.388473 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0581dd48-bf06-4f22-873a-41d96e3c5e3a admin admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:22.389095 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0581dd48-bf06-4f22-873a-41d96e3c5e3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:22.391016 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0581dd48-bf06-4f22-873a-41d96e3c5e3a admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:22.423087 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 948/1909] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:47:22 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:47:22.425225 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b55de9-4ac0-47e7-b2be-058826d773c7 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:22.435506 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b55de9-4ac0-47e7-b2be-058826d773c7 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:22.494424 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cb4f07-de80-40ad-b382-97b3bbe65121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:22.495026 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cb4f07-de80-40ad-b382-97b3bbe65121 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:22.495306 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cb4f07-de80-40ad-b382-97b3bbe65121 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:22.495610 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cb4f07-de80-40ad-b382-97b3bbe65121 admin admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:22.496228 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cb4f07-de80-40ad-b382-97b3bbe65121 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:22.498165 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cb4f07-de80-40ad-b382-97b3bbe65121 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:22.519223 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 962/1910] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:47:22 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:47:22.523374 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0581dd48-bf06-4f22-873a-41d96e3c5e3a admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:22.534304 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0581dd48-bf06-4f22-873a-41d96e3c5e3a admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:22.538958 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645c540e-30f3-4b33-9a29-3f781a6b87f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:22.539653 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645c540e-30f3-4b33-9a29-3f781a6b87f7 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:22.539888 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645c540e-30f3-4b33-9a29-3f781a6b87f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:22.540162 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645c540e-30f3-4b33-9a29-3f781a6b87f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:22.540799 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645c540e-30f3-4b33-9a29-3f781a6b87f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:22.542389 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645c540e-30f3-4b33-9a29-3f781a6b87f7 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:22.552768 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 949/1911] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:47:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 10:47:22.556603 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cb4f07-de80-40ad-b382-97b3bbe65121 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:22.566411 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cb4f07-de80-40ad-b382-97b3bbe65121 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:22.570909 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c36c5a3-fa4f-498f-99ae-a0466ed9ce9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:22.571533 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c36c5a3-fa4f-498f-99ae-a0466ed9ce9d admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:22.571817 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c36c5a3-fa4f-498f-99ae-a0466ed9ce9d admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:22.572092 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c36c5a3-fa4f-498f-99ae-a0466ed9ce9d admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:22.572727 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c36c5a3-fa4f-498f-99ae-a0466ed9ce9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:22.574307 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c36c5a3-fa4f-498f-99ae-a0466ed9ce9d admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:22.584708 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 963/1912] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:47:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:47:22.588915 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645c540e-30f3-4b33-9a29-3f781a6b87f7 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:22.599205 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645c540e-30f3-4b33-9a29-3f781a6b87f7 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:22.603793 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da126186-e84d-4b67-a6bc-85b2dd9285e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:22.604434 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da126186-e84d-4b67-a6bc-85b2dd9285e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:22.604718 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da126186-e84d-4b67-a6bc-85b2dd9285e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:22.604994 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da126186-e84d-4b67-a6bc-85b2dd9285e8 admin admin] PATH_INFO: `/v3/projects/072a17fe2391408996ec02c5723f04ee/users/ef7423bd14644dcca25142af6392ac1c/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:22.620248 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da126186-e84d-4b67-a6bc-85b2dd9285e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=072a17fe2391408996ec02c5723f04ee, user_id=ef7423bd14644dcca25142af6392ac1c, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:22.622880 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da126186-e84d-4b67-a6bc-85b2dd9285e8 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:22.636494 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 950/1913] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:47:22 2022] PUT /identity/v3/projects/072a17fe2391408996ec02c5723f04ee/users/ef7423bd14644dcca25142af6392ac1c/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:47:22.646678 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0f146c-10a3-42df-bb34-8b066906ebd1 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:22.646963 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0f146c-10a3-42df-bb34-8b066906ebd1 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:22.647263 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0f146c-10a3-42df-bb34-8b066906ebd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:22.678564 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-4e0f146c-10a3-42df-bb34-8b066906ebd1 None None] MFA Rules not processed for user `ef7423bd14644dcca25142af6392ac1c`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:47:22.679809 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-4e0f146c-10a3-42df-bb34-8b066906ebd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:22.753133 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 964/1914] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:47:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:47:22.764045 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da126186-e84d-4b67-a6bc-85b2dd9285e8 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:22.765745 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-da126186-e84d-4b67-a6bc-85b2dd9285e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:22.844515 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:22.867768 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da126186-e84d-4b67-a6bc-85b2dd9285e8 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:22.871711 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b373270b-5e05-464c-b1c2-166b2a6aad5f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1632da5fbe384b5c849be1d4d86c0f1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:22.872063 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b373270b-5e05-464c-b1c2-166b2a6aad5f service glance] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:22.872404 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b373270b-5e05-464c-b1c2-166b2a6aad5f service glance] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:22.873225 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b373270b-5e05-464c-b1c2-166b2a6aad5f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:22.883052 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b373270b-5e05-464c-b1c2-166b2a6aad5f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:22.884536 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b373270b-5e05-464c-b1c2-166b2a6aad5f service glance] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:22.894687 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 951/1915] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:47:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:47:25.588957 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0f146c-10a3-42df-bb34-8b066906ebd1 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:25.590988 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-4e0f146c-10a3-42df-bb34-8b066906ebd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:25.601044 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:25.674051 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0f146c-10a3-42df-bb34-8b066906ebd1 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:25.680892 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5a2cd7-dadd-4ba8-8744-faa3987339cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:25.681996 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5a2cd7-dadd-4ba8-8744-faa3987339cc service cinder] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:25.682493 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5a2cd7-dadd-4ba8-8744-faa3987339cc service cinder] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:25.682775 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5a2cd7-dadd-4ba8-8744-faa3987339cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:25.685427 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-5b5a2cd7-dadd-4ba8-8744-faa3987339cc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:25.730612 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b373270b-5e05-464c-b1c2-166b2a6aad5f service glance] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:25.732774 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-b373270b-5e05-464c-b1c2-166b2a6aad5f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:25.748735 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b373270b-5e05-464c-b1c2-166b2a6aad5f service glance] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:25.755268 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329be8c3-1741-460c-8376-c87dc9afc2ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:25.755908 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329be8c3-1741-460c-8376-c87dc9afc2ab service cinder] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:25.756192 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329be8c3-1741-460c-8376-c87dc9afc2ab service cinder] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:25.756464 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329be8c3-1741-460c-8376-c87dc9afc2ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:25.758215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-329be8c3-1741-460c-8376-c87dc9afc2ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:25.767013 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5a2cd7-dadd-4ba8-8744-faa3987339cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:25.769609 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5a2cd7-dadd-4ba8-8744-faa3987339cc service cinder] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:25.781074 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 965/1916] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:47:25 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:47:25.844152 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329be8c3-1741-460c-8376-c87dc9afc2ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:25.845718 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329be8c3-1741-460c-8376-c87dc9afc2ab service cinder] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:25.859144 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 952/1917] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:47:25 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 10:47:26.350863 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5a2cd7-dadd-4ba8-8744-faa3987339cc service cinder] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:26.352320 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-5b5a2cd7-dadd-4ba8-8744-faa3987339cc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:26.419680 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5a2cd7-dadd-4ba8-8744-faa3987339cc service cinder] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:26.426361 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85da6642-685c-44d6-805a-4d9cb56b1b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:26.427998 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da6642-685c-44d6-805a-4d9cb56b1b91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:26.428286 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da6642-685c-44d6-805a-4d9cb56b1b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:26.428569 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da6642-685c-44d6-805a-4d9cb56b1b91 admin admin] PATH_INFO: `/v3/users/d4fa5cef363b406da7a207f4e4bf3ccc` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:26.429854 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85da6642-685c-44d6-805a-4d9cb56b1b91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4fa5cef363b406da7a207f4e4bf3ccc)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:26.431958 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85da6642-685c-44d6-805a-4d9cb56b1b91 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:26.492729 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-85da6642-685c-44d6-805a-4d9cb56b1b91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4fa5cef363b406da7a207f4e4bf3ccc'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:26.508117 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-85da6642-685c-44d6-805a-4d9cb56b1b91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4fa5cef363b406da7a207f4e4bf3ccc'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:26.509195 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-85da6642-685c-44d6-805a-4d9cb56b1b91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4fa5cef363b406da7a207f4e4bf3ccc'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:26.509865 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-85da6642-685c-44d6-805a-4d9cb56b1b91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4fa5cef363b406da7a207f4e4bf3ccc'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:26.512112 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:26.538750 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-85da6642-685c-44d6-805a-4d9cb56b1b91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4fa5cef363b406da7a207f4e4bf3ccc'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:26.554821 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 966/1918] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:47:26 2022] DELETE /identity/v3/users/d4fa5cef363b406da7a207f4e4bf3ccc => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:47:26.558991 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f808f1-6e80-474a-bb73-4ec4dab11a3d None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:26.559360 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f808f1-6e80-474a-bb73-4ec4dab11a3d None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:26.559734 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f808f1-6e80-474a-bb73-4ec4dab11a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:26.590556 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-35f808f1-6e80-474a-bb73-4ec4dab11a3d None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:47:26.592310 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-35f808f1-6e80-474a-bb73-4ec4dab11a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:26.662188 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 953/1919] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:47:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:47:26.699881 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85da6642-685c-44d6-805a-4d9cb56b1b91 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:26.701694 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-85da6642-685c-44d6-805a-4d9cb56b1b91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:26.770154 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85da6642-685c-44d6-805a-4d9cb56b1b91 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:26.781211 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37546d0e-15f1-4092-a8de-a2ae85e526c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:26.782653 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37546d0e-15f1-4092-a8de-a2ae85e526c5 service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:26.783651 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37546d0e-15f1-4092-a8de-a2ae85e526c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:26.784059 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37546d0e-15f1-4092-a8de-a2ae85e526c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:26.796571 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37546d0e-15f1-4092-a8de-a2ae85e526c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:26.798510 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37546d0e-15f1-4092-a8de-a2ae85e526c5 service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:26.811055 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 967/1920] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:47:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:47:27.231568 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f808f1-6e80-474a-bb73-4ec4dab11a3d None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:27.233704 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-35f808f1-6e80-474a-bb73-4ec4dab11a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:27.255820 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f808f1-6e80-474a-bb73-4ec4dab11a3d None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:27.262112 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0678e6a9-c79e-4285-bc65-07687e14e08a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:27.262754 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0678e6a9-c79e-4285-bc65-07687e14e08a admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:27.263032 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0678e6a9-c79e-4285-bc65-07687e14e08a admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:27.263298 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0678e6a9-c79e-4285-bc65-07687e14e08a admin admin] PATH_INFO: `/v3/projects/57a8607224b74110ad15ca9a8deacabe` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:27.264529 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0678e6a9-c79e-4285-bc65-07687e14e08a admin admin] RBAC: Authorizing `identity:delete_project(project_id=57a8607224b74110ad15ca9a8deacabe)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:27.266702 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0678e6a9-c79e-4285-bc65-07687e14e08a admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:27.328953 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-0678e6a9-c79e-4285-bc65-07687e14e08a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57a8607224b74110ad15ca9a8deacabe', 'request_id': 'req-0678e6a9-c79e-4285-bc65-07687e14e08a'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:27.339792 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-0678e6a9-c79e-4285-bc65-07687e14e08a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57a8607224b74110ad15ca9a8deacabe', 'request_id': 'req-0678e6a9-c79e-4285-bc65-07687e14e08a'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:27.357452 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-0678e6a9-c79e-4285-bc65-07687e14e08a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57a8607224b74110ad15ca9a8deacabe', 'request_id': 'req-0678e6a9-c79e-4285-bc65-07687e14e08a'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:27.369139 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:27.377729 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-0678e6a9-c79e-4285-bc65-07687e14e08a admin admin] The token cache is being invalidate because project 57a8607224b74110ad15ca9a8deacabe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90653) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:47:27.378786 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-0678e6a9-c79e-4285-bc65-07687e14e08a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:27.379748 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-0678e6a9-c79e-4285-bc65-07687e14e08a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:27.381438 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 954/1921] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:47:27 2022] DELETE /identity/v3/projects/57a8607224b74110ad15ca9a8deacabe => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:47:27.385644 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37546d0e-15f1-4092-a8de-a2ae85e526c5 service neutron] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:27.396706 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37546d0e-15f1-4092-a8de-a2ae85e526c5 service neutron] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:27.477372 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d7cf21-5f18-4e94-ad5a-45bfe90caad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:27.477975 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d7cf21-5f18-4e94-ad5a-45bfe90caad9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:27.478246 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d7cf21-5f18-4e94-ad5a-45bfe90caad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:27.478521 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d7cf21-5f18-4e94-ad5a-45bfe90caad9 admin admin] PATH_INFO: `/v3/users/2986e5cce18d49cbb8156e98f5cf0725` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:27.479696 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d7cf21-5f18-4e94-ad5a-45bfe90caad9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2986e5cce18d49cbb8156e98f5cf0725)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:27.481875 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d7cf21-5f18-4e94-ad5a-45bfe90caad9 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:27.540108 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-35d7cf21-5f18-4e94-ad5a-45bfe90caad9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2986e5cce18d49cbb8156e98f5cf0725'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:27.549673 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-35d7cf21-5f18-4e94-ad5a-45bfe90caad9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2986e5cce18d49cbb8156e98f5cf0725'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:27.550429 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-35d7cf21-5f18-4e94-ad5a-45bfe90caad9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2986e5cce18d49cbb8156e98f5cf0725'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:27.551833 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-35d7cf21-5f18-4e94-ad5a-45bfe90caad9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2986e5cce18d49cbb8156e98f5cf0725'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:27.566600 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-35d7cf21-5f18-4e94-ad5a-45bfe90caad9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2986e5cce18d49cbb8156e98f5cf0725'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:27.571229 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:27.584028 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 968/1922] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:47:27 2022] DELETE /identity/v3/users/2986e5cce18d49cbb8156e98f5cf0725 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:47:27.788966 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3289f84-8114-4f7c-a187-a48642f24f6f None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:27.789257 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3289f84-8114-4f7c-a187-a48642f24f6f None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:27.789720 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3289f84-8114-4f7c-a187-a48642f24f6f None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:27.791396 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 955/1923] 10.222.0.15 () {54 vars in 921 bytes} [Sun Sep 11 10:47:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:47:27.803883 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a007713e-b9bc-4724-877e-4f46012d53a3 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:27.804328 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a007713e-b9bc-4724-877e-4f46012d53a3 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:27.804328 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a007713e-b9bc-4724-877e-4f46012d53a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:27.828975 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-a007713e-b9bc-4724-877e-4f46012d53a3 None None] MFA Rules not processed for user `2bd5356829b6454993bab9bad2f79fed`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:47:27.830273 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-a007713e-b9bc-4724-877e-4f46012d53a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:27.917313 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 969/1924] 10.222.0.15 () {58 vars in 1018 bytes} [Sun Sep 11 10:47:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:47:27.919481 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3289f84-8114-4f7c-a187-a48642f24f6f None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:27.921624 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-e3289f84-8114-4f7c-a187-a48642f24f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:27.937653 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a007713e-b9bc-4724-877e-4f46012d53a3 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:27.941204 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-a007713e-b9bc-4724-877e-4f46012d53a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:27.961897 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a007713e-b9bc-4724-877e-4f46012d53a3 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:27.975854 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fc1182-5f1b-4b5b-857a-6c2e30a06171 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bd5356829b6454993bab9bad2f79fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:27.976899 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fc1182-5f1b-4b5b-857a-6c2e30a06171 service nova] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:27.977421 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fc1182-5f1b-4b5b-857a-6c2e30a06171 service nova] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:27.977773 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fc1182-5f1b-4b5b-857a-6c2e30a06171 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:27.990016 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fc1182-5f1b-4b5b-857a-6c2e30a06171 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:27.991804 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fc1182-5f1b-4b5b-857a-6c2e30a06171 service nova] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:28.016688 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 970/1925] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:47:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 10:47:28.040420 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3289f84-8114-4f7c-a187-a48642f24f6f None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:28.045381 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c658ca5-4e60-4bfe-842c-3828fb836296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:28.046468 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c658ca5-4e60-4bfe-842c-3828fb836296 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:28.046971 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c658ca5-4e60-4bfe-842c-3828fb836296 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:28.047480 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c658ca5-4e60-4bfe-842c-3828fb836296 admin admin] PATH_INFO: `/v3/projects/44405dd11c564907a56ba7be0e027768` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:28.049749 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c658ca5-4e60-4bfe-842c-3828fb836296 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44405dd11c564907a56ba7be0e027768)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:28.051975 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c658ca5-4e60-4bfe-842c-3828fb836296 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:28.114685 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-7c658ca5-4e60-4bfe-842c-3828fb836296 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44405dd11c564907a56ba7be0e027768', 'request_id': 'req-7c658ca5-4e60-4bfe-842c-3828fb836296'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:28.121821 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-7c658ca5-4e60-4bfe-842c-3828fb836296 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44405dd11c564907a56ba7be0e027768', 'request_id': 'req-7c658ca5-4e60-4bfe-842c-3828fb836296'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:28.140891 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-7c658ca5-4e60-4bfe-842c-3828fb836296 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44405dd11c564907a56ba7be0e027768', 'request_id': 'req-7c658ca5-4e60-4bfe-842c-3828fb836296'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:28.154213 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-7c658ca5-4e60-4bfe-842c-3828fb836296 admin admin] The token cache is being invalidate because project 44405dd11c564907a56ba7be0e027768 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90653) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:47:28.154581 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-7c658ca5-4e60-4bfe-842c-3828fb836296 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:28.155352 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-7c658ca5-4e60-4bfe-842c-3828fb836296 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:28.156987 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 956/1926] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:47:27 2022] DELETE /identity/v3/projects/44405dd11c564907a56ba7be0e027768 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:47:28.162694 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af28f5-d578-48f9-a818-538cff987a83 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:28.163320 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af28f5-d578-48f9-a818-538cff987a83 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:28.163650 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af28f5-d578-48f9-a818-538cff987a83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:28.185629 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-00af28f5-d578-48f9-a818-538cff987a83 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:47:28.186863 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-00af28f5-d578-48f9-a818-538cff987a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:28.253469 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 971/1927] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:47:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 10:47:28.262521 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce144f1-c04c-4b53-911e-ce361e79d98b None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:28.263365 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce144f1-c04c-4b53-911e-ce361e79d98b None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:28.263662 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce144f1-c04c-4b53-911e-ce361e79d98b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:28.286417 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-3ce144f1-c04c-4b53-911e-ce361e79d98b None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:47:28.287876 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-3ce144f1-c04c-4b53-911e-ce361e79d98b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:28.296485 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 957/1928] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:47:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:47:28.301228 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af28f5-d578-48f9-a818-538cff987a83 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:28.311178 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af28f5-d578-48f9-a818-538cff987a83 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:28.316671 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de522b68-37be-418b-8d7b-3bd3423c8548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:28.317365 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de522b68-37be-418b-8d7b-3bd3423c8548 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:28.317753 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de522b68-37be-418b-8d7b-3bd3423c8548 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:28.318378 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de522b68-37be-418b-8d7b-3bd3423c8548 admin admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:28.318837 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de522b68-37be-418b-8d7b-3bd3423c8548 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:47:28.319249 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de522b68-37be-418b-8d7b-3bd3423c8548 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:28.320989 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de522b68-37be-418b-8d7b-3bd3423c8548 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:28.334342 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 972/1929] 10.222.0.15 () {58 vars in 1153 bytes} [Sun Sep 11 10:47:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:47:28.338855 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce144f1-c04c-4b53-911e-ce361e79d98b None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:28.352173 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce144f1-c04c-4b53-911e-ce361e79d98b None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:28.357579 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f9e2b-e389-4c70-83b1-a88d9750fb79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:28.358329 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f9e2b-e389-4c70-83b1-a88d9750fb79 admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:28.358796 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f9e2b-e389-4c70-83b1-a88d9750fb79 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:28.359267 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f9e2b-e389-4c70-83b1-a88d9750fb79 admin admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:28.360107 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18f9e2b-e389-4c70-83b1-a88d9750fb79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:28.363969 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18f9e2b-e389-4c70-83b1-a88d9750fb79 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:28.393978 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 958/1930] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:47:28 2022] POST /identity/v3/projects => generated 381 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 10:47:28.398118 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de522b68-37be-418b-8d7b-3bd3423c8548 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:28.408988 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de522b68-37be-418b-8d7b-3bd3423c8548 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:28.466805 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071fe30b-bf38-45ac-999b-611984c8e6a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:28.467443 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071fe30b-bf38-45ac-999b-611984c8e6a2 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:28.467730 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071fe30b-bf38-45ac-999b-611984c8e6a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:28.468005 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071fe30b-bf38-45ac-999b-611984c8e6a2 admin admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:28.468641 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071fe30b-bf38-45ac-999b-611984c8e6a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:28.471171 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071fe30b-bf38-45ac-999b-611984c8e6a2 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:28.479291 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:28.493118 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 973/1931] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:47:28 2022] POST /identity/v3/users => generated 291 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:47:28.497226 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f9e2b-e389-4c70-83b1-a88d9750fb79 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:28.508240 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f9e2b-e389-4c70-83b1-a88d9750fb79 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:28.512755 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc74265-3ef6-4850-ac4f-c2660f4e5f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:28.513337 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc74265-3ef6-4850-ac4f-c2660f4e5f4f admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:28.513613 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc74265-3ef6-4850-ac4f-c2660f4e5f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:28.513884 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc74265-3ef6-4850-ac4f-c2660f4e5f4f admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:28.514513 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc74265-3ef6-4850-ac4f-c2660f4e5f4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:28.516206 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc74265-3ef6-4850-ac4f-c2660f4e5f4f admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:28.527236 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 959/1932] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:47:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:47:28.530946 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071fe30b-bf38-45ac-999b-611984c8e6a2 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:28.540963 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071fe30b-bf38-45ac-999b-611984c8e6a2 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:28.545537 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc61f2e-0dc1-4e2b-8a9c-dc5ced725d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:28.546132 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc61f2e-0dc1-4e2b-8a9c-dc5ced725d10 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:28.546412 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc61f2e-0dc1-4e2b-8a9c-dc5ced725d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:28.546683 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc61f2e-0dc1-4e2b-8a9c-dc5ced725d10 admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:28.547343 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc61f2e-0dc1-4e2b-8a9c-dc5ced725d10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:28.548962 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc61f2e-0dc1-4e2b-8a9c-dc5ced725d10 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:28.559303 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 974/1933] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:47:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:47:28.563282 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc74265-3ef6-4850-ac4f-c2660f4e5f4f admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:28.573564 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc74265-3ef6-4850-ac4f-c2660f4e5f4f admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:28.578344 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814cc678-8bfd-4563-9b2c-d92a80546469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:28.578969 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814cc678-8bfd-4563-9b2c-d92a80546469 admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:28.579253 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814cc678-8bfd-4563-9b2c-d92a80546469 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:28.579575 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814cc678-8bfd-4563-9b2c-d92a80546469 admin admin] PATH_INFO: `/v3/projects/3eba92288dc64d98ac98c49abda89200/users/a426c116a98c4f88aba7c4156a59fdb2/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:28.595006 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814cc678-8bfd-4563-9b2c-d92a80546469 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3eba92288dc64d98ac98c49abda89200, user_id=a426c116a98c4f88aba7c4156a59fdb2, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:28.598178 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814cc678-8bfd-4563-9b2c-d92a80546469 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:28.612573 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 960/1934] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:47:28 2022] PUT /identity/v3/projects/3eba92288dc64d98ac98c49abda89200/users/a426c116a98c4f88aba7c4156a59fdb2/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:47:28.622680 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ca735-b0df-43f0-9ef5-9302e063eecc None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:28.622964 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ca735-b0df-43f0-9ef5-9302e063eecc None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:28.623248 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ca735-b0df-43f0-9ef5-9302e063eecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:28.651752 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-e97ca735-b0df-43f0-9ef5-9302e063eecc None None] MFA Rules not processed for user `a426c116a98c4f88aba7c4156a59fdb2`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:47:28.652963 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-e97ca735-b0df-43f0-9ef5-9302e063eecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:28.725294 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 975/1935] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:47:28 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:47:28.729701 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814cc678-8bfd-4563-9b2c-d92a80546469 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:28.756484 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:28.795335 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814cc678-8bfd-4563-9b2c-d92a80546469 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:28.800232 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642292a5-9d74-47ef-8bc0-de99183606ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:28.800842 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642292a5-9d74-47ef-8bc0-de99183606ee admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:28.801128 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642292a5-9d74-47ef-8bc0-de99183606ee admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:28.801409 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642292a5-9d74-47ef-8bc0-de99183606ee admin admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:28.802036 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642292a5-9d74-47ef-8bc0-de99183606ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:28.803989 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642292a5-9d74-47ef-8bc0-de99183606ee admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:28.830696 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 961/1936] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:47:28 2022] POST /identity/v3/projects => generated 381 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:47:28.834809 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97ca735-b0df-43f0-9ef5-9302e063eecc None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:28.845485 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97ca735-b0df-43f0-9ef5-9302e063eecc None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:28.902948 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b829c6-578e-44a0-bc2d-41116d299839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:28.903626 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b829c6-578e-44a0-bc2d-41116d299839 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:28.903908 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b829c6-578e-44a0-bc2d-41116d299839 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:28.904197 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b829c6-578e-44a0-bc2d-41116d299839 admin admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:28.904984 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b829c6-578e-44a0-bc2d-41116d299839 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:28.906901 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b829c6-578e-44a0-bc2d-41116d299839 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:28.927923 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 976/1937] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:47:28 2022] POST /identity/v3/users => generated 297 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:47:28.932386 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642292a5-9d74-47ef-8bc0-de99183606ee admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:28.942706 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642292a5-9d74-47ef-8bc0-de99183606ee admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:28.947571 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902251f5-c905-4152-82d5-141537807a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:28.948168 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902251f5-c905-4152-82d5-141537807a06 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:28.948473 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902251f5-c905-4152-82d5-141537807a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:28.948745 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902251f5-c905-4152-82d5-141537807a06 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:28.949406 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902251f5-c905-4152-82d5-141537807a06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:28.951005 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902251f5-c905-4152-82d5-141537807a06 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:28.961900 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 962/1938] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:47:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:47:28.966121 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b829c6-578e-44a0-bc2d-41116d299839 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:28.976767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b829c6-578e-44a0-bc2d-41116d299839 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:28.981398 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b17a74-8c08-479e-b611-b04f4148a819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:28.982300 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b17a74-8c08-479e-b611-b04f4148a819 admin admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:28.982580 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b17a74-8c08-479e-b611-b04f4148a819 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:28.982853 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b17a74-8c08-479e-b611-b04f4148a819 admin admin] PATH_INFO: `/v3/projects/7cbc5b51e9214afab95422c3c9d5040e/users/15f4ec29ce19429dae7b193f7e76635e/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:28.997585 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b17a74-8c08-479e-b611-b04f4148a819 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cbc5b51e9214afab95422c3c9d5040e, user_id=15f4ec29ce19429dae7b193f7e76635e, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:29.000265 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b17a74-8c08-479e-b611-b04f4148a819 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:29.013305 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 977/1939] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:47:28 2022] PUT /identity/v3/projects/7cbc5b51e9214afab95422c3c9d5040e/users/15f4ec29ce19429dae7b193f7e76635e/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:47:29.017856 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902251f5-c905-4152-82d5-141537807a06 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:29.028742 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902251f5-c905-4152-82d5-141537807a06 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:29.086925 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad83f3c4-fe50-47c7-a624-016a892c28c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:29.087524 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad83f3c4-fe50-47c7-a624-016a892c28c0 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:29.087802 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad83f3c4-fe50-47c7-a624-016a892c28c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:29.088070 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad83f3c4-fe50-47c7-a624-016a892c28c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:29.088752 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad83f3c4-fe50-47c7-a624-016a892c28c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:29.090439 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad83f3c4-fe50-47c7-a624-016a892c28c0 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:29.100868 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 963/1940] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:47:29 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:47:29.105189 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b17a74-8c08-479e-b611-b04f4148a819 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:29.115825 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b17a74-8c08-479e-b611-b04f4148a819 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:29.120683 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a26eb47-4e94-49c7-a03a-f54e91bdc34b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:29.121351 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a26eb47-4e94-49c7-a03a-f54e91bdc34b admin admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:29.121632 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a26eb47-4e94-49c7-a03a-f54e91bdc34b admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:29.121905 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a26eb47-4e94-49c7-a03a-f54e91bdc34b admin admin] PATH_INFO: `/v3/projects/7cbc5b51e9214afab95422c3c9d5040e/users/15f4ec29ce19429dae7b193f7e76635e/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:29.123887 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a26eb47-4e94-49c7-a03a-f54e91bdc34b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cbc5b51e9214afab95422c3c9d5040e, user_id=15f4ec29ce19429dae7b193f7e76635e, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:29.126572 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a26eb47-4e94-49c7-a03a-f54e91bdc34b admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:29.137297 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 978/1941] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:47:29 2022] PUT /identity/v3/projects/7cbc5b51e9214afab95422c3c9d5040e/users/15f4ec29ce19429dae7b193f7e76635e/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:47:29.147802 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27918b-e18c-4f1b-afe7-714f366471c1 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:29.148120 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27918b-e18c-4f1b-afe7-714f366471c1 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:29.148373 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27918b-e18c-4f1b-afe7-714f366471c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:29.177512 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-5d27918b-e18c-4f1b-afe7-714f366471c1 None None] MFA Rules not processed for user `15f4ec29ce19429dae7b193f7e76635e`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:47:29.178722 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-5d27918b-e18c-4f1b-afe7-714f366471c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:29.250885 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 964/1942] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:47:29 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:47:29.262374 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a26eb47-4e94-49c7-a03a-f54e91bdc34b admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:29.264117 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-5a26eb47-4e94-49c7-a03a-f54e91bdc34b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:29.293001 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:29.346938 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a26eb47-4e94-49c7-a03a-f54e91bdc34b admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:29.350532 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78922d53-171d-4fad-8d3a-d25569a1ec13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:29.351153 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78922d53-171d-4fad-8d3a-d25569a1ec13 service cinder] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:29.351453 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78922d53-171d-4fad-8d3a-d25569a1ec13 service cinder] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:29.351738 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78922d53-171d-4fad-8d3a-d25569a1ec13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:29.362188 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78922d53-171d-4fad-8d3a-d25569a1ec13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:29.363658 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78922d53-171d-4fad-8d3a-d25569a1ec13 service cinder] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:29.373913 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 979/1943] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:47:29 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:47:29.468628 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d27918b-e18c-4f1b-afe7-714f366471c1 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:29.478781 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d27918b-e18c-4f1b-afe7-714f366471c1 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:29.483595 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998cc3eb-506a-4cc9-9f2d-b85c423d54fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:29.484405 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998cc3eb-506a-4cc9-9f2d-b85c423d54fc service cinder] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:29.485068 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998cc3eb-506a-4cc9-9f2d-b85c423d54fc service cinder] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:29.485522 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998cc3eb-506a-4cc9-9f2d-b85c423d54fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:29.522603 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:29.575123 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998cc3eb-506a-4cc9-9f2d-b85c423d54fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:29.576861 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998cc3eb-506a-4cc9-9f2d-b85c423d54fc service cinder] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:29.586955 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 965/1944] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:47:29 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:47:29.869232 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78922d53-171d-4fad-8d3a-d25569a1ec13 service cinder] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:29.930142 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78922d53-171d-4fad-8d3a-d25569a1ec13 service cinder] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:29.934547 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f77648-61b3-4b2d-99ce-09dd76eb9dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:29.935168 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f77648-61b3-4b2d-99ce-09dd76eb9dd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:29.935470 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f77648-61b3-4b2d-99ce-09dd76eb9dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:29.935744 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f77648-61b3-4b2d-99ce-09dd76eb9dd9 admin admin] PATH_INFO: `/v3/users/a426c116a98c4f88aba7c4156a59fdb2` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:29.936861 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f77648-61b3-4b2d-99ce-09dd76eb9dd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a426c116a98c4f88aba7c4156a59fdb2)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:29.938760 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f77648-61b3-4b2d-99ce-09dd76eb9dd9 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:29.983191 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-31f77648-61b3-4b2d-99ce-09dd76eb9dd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a426c116a98c4f88aba7c4156a59fdb2'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:29.992042 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-31f77648-61b3-4b2d-99ce-09dd76eb9dd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a426c116a98c4f88aba7c4156a59fdb2'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:29.992662 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-31f77648-61b3-4b2d-99ce-09dd76eb9dd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a426c116a98c4f88aba7c4156a59fdb2'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:29.993269 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-31f77648-61b3-4b2d-99ce-09dd76eb9dd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a426c116a98c4f88aba7c4156a59fdb2'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:30.005256 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-31f77648-61b3-4b2d-99ce-09dd76eb9dd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a426c116a98c4f88aba7c4156a59fdb2'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:30.017036 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 980/1945] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:47:29 2022] DELETE /identity/v3/users/a426c116a98c4f88aba7c4156a59fdb2 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:47:30.021514 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e12ac-3f86-4098-b302-af779abb654c None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:30.021802 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e12ac-3f86-4098-b302-af779abb654c None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:30.022106 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e12ac-3f86-4098-b302-af779abb654c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:30.041696 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-eb3e12ac-3f86-4098-b302-af779abb654c None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:47:30.042945 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-eb3e12ac-3f86-4098-b302-af779abb654c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:30.107023 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 966/1946] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:47:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:47:30.117382 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f77648-61b3-4b2d-99ce-09dd76eb9dd9 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:30.118828 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-31f77648-61b3-4b2d-99ce-09dd76eb9dd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:30.163741 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:30.176638 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f77648-61b3-4b2d-99ce-09dd76eb9dd9 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:30.180485 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526ba9cf-a4ac-47ba-b473-cd50fc00d309 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:30.181092 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526ba9cf-a4ac-47ba-b473-cd50fc00d309 service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:30.181371 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526ba9cf-a4ac-47ba-b473-cd50fc00d309 service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:30.181644 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526ba9cf-a4ac-47ba-b473-cd50fc00d309 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:30.190775 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526ba9cf-a4ac-47ba-b473-cd50fc00d309 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:30.192232 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526ba9cf-a4ac-47ba-b473-cd50fc00d309 service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:30.201440 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 981/1947] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:47:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:47:30.477879 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3e12ac-3f86-4098-b302-af779abb654c None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:30.479320 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-eb3e12ac-3f86-4098-b302-af779abb654c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:30.493781 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3e12ac-3f86-4098-b302-af779abb654c None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:30.498919 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701f2728-3951-43d1-98d2-b398851fea26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:30.499602 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701f2728-3951-43d1-98d2-b398851fea26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:30.499934 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701f2728-3951-43d1-98d2-b398851fea26 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:30.500234 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701f2728-3951-43d1-98d2-b398851fea26 admin admin] PATH_INFO: `/v3/projects/3eba92288dc64d98ac98c49abda89200` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:30.501438 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701f2728-3951-43d1-98d2-b398851fea26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3eba92288dc64d98ac98c49abda89200)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:30.504281 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701f2728-3951-43d1-98d2-b398851fea26 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:30.561523 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-701f2728-3951-43d1-98d2-b398851fea26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3eba92288dc64d98ac98c49abda89200', 'request_id': 'req-701f2728-3951-43d1-98d2-b398851fea26'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:30.568661 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-701f2728-3951-43d1-98d2-b398851fea26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3eba92288dc64d98ac98c49abda89200', 'request_id': 'req-701f2728-3951-43d1-98d2-b398851fea26'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:30.577613 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-701f2728-3951-43d1-98d2-b398851fea26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3eba92288dc64d98ac98c49abda89200', 'request_id': 'req-701f2728-3951-43d1-98d2-b398851fea26'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:30.590422 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-701f2728-3951-43d1-98d2-b398851fea26 admin admin] The token cache is being invalidate because project 3eba92288dc64d98ac98c49abda89200 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90653) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:47:30.590709 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-701f2728-3951-43d1-98d2-b398851fea26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:30.591395 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-701f2728-3951-43d1-98d2-b398851fea26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:30.593413 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 967/1948] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:47:30 2022] DELETE /identity/v3/projects/3eba92288dc64d98ac98c49abda89200 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 10:47:30.598533 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526ba9cf-a4ac-47ba-b473-cd50fc00d309 service neutron] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:30.608961 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526ba9cf-a4ac-47ba-b473-cd50fc00d309 service neutron] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:30.672300 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f78ef-8ddb-4b83-8f21-7a559188ea3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:30.672946 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f78ef-8ddb-4b83-8f21-7a559188ea3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:30.673222 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f78ef-8ddb-4b83-8f21-7a559188ea3b admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:30.673532 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f78ef-8ddb-4b83-8f21-7a559188ea3b admin admin] PATH_INFO: `/v3/users/15f4ec29ce19429dae7b193f7e76635e` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:30.675125 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194f78ef-8ddb-4b83-8f21-7a559188ea3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=15f4ec29ce19429dae7b193f7e76635e)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:30.677085 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194f78ef-8ddb-4b83-8f21-7a559188ea3b admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:30.733420 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-194f78ef-8ddb-4b83-8f21-7a559188ea3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15f4ec29ce19429dae7b193f7e76635e'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:30.741678 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-194f78ef-8ddb-4b83-8f21-7a559188ea3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15f4ec29ce19429dae7b193f7e76635e'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:30.742363 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-194f78ef-8ddb-4b83-8f21-7a559188ea3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15f4ec29ce19429dae7b193f7e76635e'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:30.743059 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-194f78ef-8ddb-4b83-8f21-7a559188ea3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15f4ec29ce19429dae7b193f7e76635e'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:30.762860 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-194f78ef-8ddb-4b83-8f21-7a559188ea3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15f4ec29ce19429dae7b193f7e76635e'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:30.779809 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 982/1949] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:47:30 2022] DELETE /identity/v3/users/15f4ec29ce19429dae7b193f7e76635e => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:47:31.063333 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701f2728-3951-43d1-98d2-b398851fea26 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:31.065645 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-701f2728-3951-43d1-98d2-b398851fea26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:31.078664 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701f2728-3951-43d1-98d2-b398851fea26 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:31.119048 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:31.139994 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a27136-375d-4448-9dde-07f719fe55c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:31.140631 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a27136-375d-4448-9dde-07f719fe55c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:31.140911 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a27136-375d-4448-9dde-07f719fe55c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:31.141183 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a27136-375d-4448-9dde-07f719fe55c4 admin admin] PATH_INFO: `/v3/projects/7cbc5b51e9214afab95422c3c9d5040e` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:31.142322 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a27136-375d-4448-9dde-07f719fe55c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cbc5b51e9214afab95422c3c9d5040e)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:31.144344 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a27136-375d-4448-9dde-07f719fe55c4 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:31.195556 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-27a27136-375d-4448-9dde-07f719fe55c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cbc5b51e9214afab95422c3c9d5040e', 'request_id': 'req-27a27136-375d-4448-9dde-07f719fe55c4'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:31.201679 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-27a27136-375d-4448-9dde-07f719fe55c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cbc5b51e9214afab95422c3c9d5040e', 'request_id': 'req-27a27136-375d-4448-9dde-07f719fe55c4'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:31.209430 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-27a27136-375d-4448-9dde-07f719fe55c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cbc5b51e9214afab95422c3c9d5040e', 'request_id': 'req-27a27136-375d-4448-9dde-07f719fe55c4'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:31.219899 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-27a27136-375d-4448-9dde-07f719fe55c4 admin admin] The token cache is being invalidate because project 7cbc5b51e9214afab95422c3c9d5040e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90653) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:47:31.220187 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-27a27136-375d-4448-9dde-07f719fe55c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:31.220810 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-27a27136-375d-4448-9dde-07f719fe55c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:31.222341 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 968/1950] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:47:31 2022] DELETE /identity/v3/projects/7cbc5b51e9214afab95422c3c9d5040e => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:47:31.227662 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec74b5-695e-405c-8b34-a5b77ab0a365 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:31.228155 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec74b5-695e-405c-8b34-a5b77ab0a365 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:31.228345 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec74b5-695e-405c-8b34-a5b77ab0a365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:31.248973 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-29ec74b5-695e-405c-8b34-a5b77ab0a365 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:47:31.250171 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-29ec74b5-695e-405c-8b34-a5b77ab0a365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:31.308073 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 983/1951] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:47:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:47:31.315661 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b30e1-9932-4d7e-b35a-28ef3cfc1dba None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:31.315952 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b30e1-9932-4d7e-b35a-28ef3cfc1dba None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:31.316261 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b30e1-9932-4d7e-b35a-28ef3cfc1dba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:31.336418 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-519b30e1-9932-4d7e-b35a-28ef3cfc1dba None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:47:31.337621 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-519b30e1-9932-4d7e-b35a-28ef3cfc1dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:31.346112 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 969/1952] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:47:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:47:31.351012 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ec74b5-695e-405c-8b34-a5b77ab0a365 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:31.357339 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:31.361310 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ec74b5-695e-405c-8b34-a5b77ab0a365 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:31.367479 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9f72fc-a154-4965-b7bf-5523b8f08b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:31.368263 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9f72fc-a154-4965-b7bf-5523b8f08b3a admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:31.368811 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9f72fc-a154-4965-b7bf-5523b8f08b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:31.369245 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9f72fc-a154-4965-b7bf-5523b8f08b3a admin admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:31.369897 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9f72fc-a154-4965-b7bf-5523b8f08b3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:47:31.370202 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9f72fc-a154-4965-b7bf-5523b8f08b3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:31.371843 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9f72fc-a154-4965-b7bf-5523b8f08b3a admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:31.383865 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 984/1953] 10.222.0.15 () {58 vars in 1153 bytes} [Sun Sep 11 10:47:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:47:31.388382 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519b30e1-9932-4d7e-b35a-28ef3cfc1dba None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:31.398481 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519b30e1-9932-4d7e-b35a-28ef3cfc1dba None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:31.403087 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49dba52-53b7-4ce6-a9ca-f2683eaf68ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:31.403731 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49dba52-53b7-4ce6-a9ca-f2683eaf68ec admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:31.404013 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49dba52-53b7-4ce6-a9ca-f2683eaf68ec admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:31.404288 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49dba52-53b7-4ce6-a9ca-f2683eaf68ec admin admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:31.404916 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49dba52-53b7-4ce6-a9ca-f2683eaf68ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:31.406807 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49dba52-53b7-4ce6-a9ca-f2683eaf68ec admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:31.434213 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 970/1954] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:47:31 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:47:31.438508 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9f72fc-a154-4965-b7bf-5523b8f08b3a admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:31.449118 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9f72fc-a154-4965-b7bf-5523b8f08b3a admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:31.507371 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a7e857-6ad6-4079-9d8e-578927cd192e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:31.508147 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a7e857-6ad6-4079-9d8e-578927cd192e admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:31.508426 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a7e857-6ad6-4079-9d8e-578927cd192e admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:31.508698 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a7e857-6ad6-4079-9d8e-578927cd192e admin admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:31.509375 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a7e857-6ad6-4079-9d8e-578927cd192e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:31.511260 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a7e857-6ad6-4079-9d8e-578927cd192e admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:31.534523 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 985/1955] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:47:31 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:47:31.538623 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49dba52-53b7-4ce6-a9ca-f2683eaf68ec admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:31.549254 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49dba52-53b7-4ce6-a9ca-f2683eaf68ec admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:31.553820 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66320da6-1e8f-44c5-96df-37f17ac277e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:31.554427 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66320da6-1e8f-44c5-96df-37f17ac277e2 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:31.554710 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66320da6-1e8f-44c5-96df-37f17ac277e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:31.554985 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66320da6-1e8f-44c5-96df-37f17ac277e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:31.555660 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66320da6-1e8f-44c5-96df-37f17ac277e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:31.557240 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66320da6-1e8f-44c5-96df-37f17ac277e2 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:31.570751 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 971/1956] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:47:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:47:31.571924 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a7e857-6ad6-4079-9d8e-578927cd192e admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:31.582426 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a7e857-6ad6-4079-9d8e-578927cd192e admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:31.586998 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0519261e-98c2-4eeb-a89a-997cefa436b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:31.587703 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0519261e-98c2-4eeb-a89a-997cefa436b4 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:31.587989 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0519261e-98c2-4eeb-a89a-997cefa436b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:31.588260 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0519261e-98c2-4eeb-a89a-997cefa436b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:31.588901 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0519261e-98c2-4eeb-a89a-997cefa436b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:31.590514 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0519261e-98c2-4eeb-a89a-997cefa436b4 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:31.601515 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 986/1957] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:47:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 10:47:31.605633 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66320da6-1e8f-44c5-96df-37f17ac277e2 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:31.616234 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66320da6-1e8f-44c5-96df-37f17ac277e2 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:31.621200 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0115643c-35cd-4a57-9963-f4957ae6566f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:31.621876 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0115643c-35cd-4a57-9963-f4957ae6566f admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:31.622162 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0115643c-35cd-4a57-9963-f4957ae6566f admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:31.622441 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0115643c-35cd-4a57-9963-f4957ae6566f admin admin] PATH_INFO: `/v3/projects/059ded7d632a4e0e91e74d5cdde54725/users/d02554a02da54173bd4dd3200feaf392/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:31.638253 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0115643c-35cd-4a57-9963-f4957ae6566f admin admin] RBAC: Authorizing `identity:create_grant(project_id=059ded7d632a4e0e91e74d5cdde54725, user_id=d02554a02da54173bd4dd3200feaf392, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:31.641040 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0115643c-35cd-4a57-9963-f4957ae6566f admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:31.655832 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 972/1958] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:47:31 2022] PUT /identity/v3/projects/059ded7d632a4e0e91e74d5cdde54725/users/d02554a02da54173bd4dd3200feaf392/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 10:47:31.666050 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152b6676-64b1-4046-92a3-0ed35cabe1ab None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:31.666411 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152b6676-64b1-4046-92a3-0ed35cabe1ab None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:31.666689 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152b6676-64b1-4046-92a3-0ed35cabe1ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:31.697766 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-152b6676-64b1-4046-92a3-0ed35cabe1ab None None] MFA Rules not processed for user `d02554a02da54173bd4dd3200feaf392`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:47:31.699003 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-152b6676-64b1-4046-92a3-0ed35cabe1ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:31.775879 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 987/1959] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:47:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:47:31.780524 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0115643c-35cd-4a57-9963-f4957ae6566f admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:31.844332 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0115643c-35cd-4a57-9963-f4957ae6566f admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:31.849389 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacb87bd-efcc-46a2-8555-e5c3102f88cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:31.850007 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb87bd-efcc-46a2-8555-e5c3102f88cd admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:31.850287 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb87bd-efcc-46a2-8555-e5c3102f88cd admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:31.850559 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb87bd-efcc-46a2-8555-e5c3102f88cd admin admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:31.851202 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacb87bd-efcc-46a2-8555-e5c3102f88cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:31.853128 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacb87bd-efcc-46a2-8555-e5c3102f88cd admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:31.866703 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:31.889777 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 973/1960] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:47:31 2022] POST /identity/v3/projects => generated 383 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:47:31.894104 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152b6676-64b1-4046-92a3-0ed35cabe1ab None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:31.909588 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152b6676-64b1-4046-92a3-0ed35cabe1ab None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:31.984016 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae10a30d-60f0-448f-bec9-3d6f2f1936e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:31.984548 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae10a30d-60f0-448f-bec9-3d6f2f1936e2 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:31.984825 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae10a30d-60f0-448f-bec9-3d6f2f1936e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:31.985095 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae10a30d-60f0-448f-bec9-3d6f2f1936e2 admin admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:31.985720 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae10a30d-60f0-448f-bec9-3d6f2f1936e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:31.988803 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae10a30d-60f0-448f-bec9-3d6f2f1936e2 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:31.997227 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:32.015559 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 988/1961] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:47:31 2022] POST /identity/v3/users => generated 292 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:47:32.018043 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacb87bd-efcc-46a2-8555-e5c3102f88cd admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:32.029126 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacb87bd-efcc-46a2-8555-e5c3102f88cd admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:32.034948 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83796697-b732-4af5-ba9a-27dd56e11401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:32.035613 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83796697-b732-4af5-ba9a-27dd56e11401 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:32.035951 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83796697-b732-4af5-ba9a-27dd56e11401 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:32.036227 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83796697-b732-4af5-ba9a-27dd56e11401 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:32.037005 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83796697-b732-4af5-ba9a-27dd56e11401 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:32.038608 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83796697-b732-4af5-ba9a-27dd56e11401 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:32.051924 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 974/1962] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:47:32 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:47:32.055517 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae10a30d-60f0-448f-bec9-3d6f2f1936e2 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:32.066540 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae10a30d-60f0-448f-bec9-3d6f2f1936e2 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:32.071846 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74fd834-4e55-41fe-a0af-6547e8c92070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:32.072464 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74fd834-4e55-41fe-a0af-6547e8c92070 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:32.072744 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74fd834-4e55-41fe-a0af-6547e8c92070 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:32.073126 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74fd834-4e55-41fe-a0af-6547e8c92070 admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:32.074097 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74fd834-4e55-41fe-a0af-6547e8c92070 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:32.075753 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74fd834-4e55-41fe-a0af-6547e8c92070 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:32.086527 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 989/1963] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:47:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:47:32.090870 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83796697-b732-4af5-ba9a-27dd56e11401 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:32.102107 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83796697-b732-4af5-ba9a-27dd56e11401 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:32.110925 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4db4378-3857-49ff-bf9d-f0b297d1f0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:32.112605 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4db4378-3857-49ff-bf9d-f0b297d1f0f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:32.113030 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4db4378-3857-49ff-bf9d-f0b297d1f0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:32.114133 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4db4378-3857-49ff-bf9d-f0b297d1f0f9 admin admin] PATH_INFO: `/v3/projects/ee9e43e71aa9499dbc6a0b2b0807ffa3/users/2dc88790380e4c3e84f8f5f0db3400d5/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:32.129770 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4db4378-3857-49ff-bf9d-f0b297d1f0f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee9e43e71aa9499dbc6a0b2b0807ffa3, user_id=2dc88790380e4c3e84f8f5f0db3400d5, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:32.132470 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4db4378-3857-49ff-bf9d-f0b297d1f0f9 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:32.148091 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 975/1964] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:47:32 2022] PUT /identity/v3/projects/ee9e43e71aa9499dbc6a0b2b0807ffa3/users/2dc88790380e4c3e84f8f5f0db3400d5/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 10:47:32.162838 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce210bea-d0b6-44c1-bec4-7d98a50f8a51 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:32.163129 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce210bea-d0b6-44c1-bec4-7d98a50f8a51 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:32.163403 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce210bea-d0b6-44c1-bec4-7d98a50f8a51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:32.191026 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-ce210bea-d0b6-44c1-bec4-7d98a50f8a51 None None] MFA Rules not processed for user `2dc88790380e4c3e84f8f5f0db3400d5`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:47:32.192363 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-ce210bea-d0b6-44c1-bec4-7d98a50f8a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:32.260782 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 990/1965] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:47:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:47:32.265534 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4db4378-3857-49ff-bf9d-f0b297d1f0f9 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:32.331667 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4db4378-3857-49ff-bf9d-f0b297d1f0f9 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:32.336249 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b9414-2193-4665-a989-126265962d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:32.336853 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b9414-2193-4665-a989-126265962d0f admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:32.337133 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b9414-2193-4665-a989-126265962d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:32.337407 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b9414-2193-4665-a989-126265962d0f admin admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:32.338043 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723b9414-2193-4665-a989-126265962d0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:32.340030 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723b9414-2193-4665-a989-126265962d0f admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:32.368415 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 976/1966] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:47:32 2022] POST /identity/v3/projects => generated 381 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:47:32.372559 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce210bea-d0b6-44c1-bec4-7d98a50f8a51 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:32.385372 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce210bea-d0b6-44c1-bec4-7d98a50f8a51 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:32.442287 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12df3244-99f8-454b-8147-3dbf5c6d9c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:32.442905 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df3244-99f8-454b-8147-3dbf5c6d9c6c admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:32.443183 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df3244-99f8-454b-8147-3dbf5c6d9c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:32.443483 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df3244-99f8-454b-8147-3dbf5c6d9c6c admin admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:32.444118 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12df3244-99f8-454b-8147-3dbf5c6d9c6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:32.446037 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12df3244-99f8-454b-8147-3dbf5c6d9c6c admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:32.467285 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 991/1967] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:47:32 2022] POST /identity/v3/users => generated 297 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:47:32.471984 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b9414-2193-4665-a989-126265962d0f admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:32.482921 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b9414-2193-4665-a989-126265962d0f admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:32.488975 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dffb1a-24af-4e2c-be4b-d0c1f9b62334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:32.489580 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dffb1a-24af-4e2c-be4b-d0c1f9b62334 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:32.489862 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dffb1a-24af-4e2c-be4b-d0c1f9b62334 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:32.490139 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dffb1a-24af-4e2c-be4b-d0c1f9b62334 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:32.490779 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dffb1a-24af-4e2c-be4b-d0c1f9b62334 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:32.492390 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dffb1a-24af-4e2c-be4b-d0c1f9b62334 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:32.503607 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 977/1968] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:47:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 10:47:32.507361 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12df3244-99f8-454b-8147-3dbf5c6d9c6c admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:32.518597 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12df3244-99f8-454b-8147-3dbf5c6d9c6c admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:32.523687 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690cc38b-e103-48bb-9a7d-199edbbdf942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:32.524347 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690cc38b-e103-48bb-9a7d-199edbbdf942 admin admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:32.524628 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690cc38b-e103-48bb-9a7d-199edbbdf942 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:32.524904 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690cc38b-e103-48bb-9a7d-199edbbdf942 admin admin] PATH_INFO: `/v3/projects/79789607cba2404eae8e49fb4bc3a5e8/users/8ca90624010448a294c3651f752e209a/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:32.540975 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690cc38b-e103-48bb-9a7d-199edbbdf942 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79789607cba2404eae8e49fb4bc3a5e8, user_id=8ca90624010448a294c3651f752e209a, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:32.543626 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690cc38b-e103-48bb-9a7d-199edbbdf942 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:32.556228 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 992/1969] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:47:32 2022] PUT /identity/v3/projects/79789607cba2404eae8e49fb4bc3a5e8/users/8ca90624010448a294c3651f752e209a/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:47:32.560143 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dffb1a-24af-4e2c-be4b-d0c1f9b62334 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:32.570365 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dffb1a-24af-4e2c-be4b-d0c1f9b62334 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:32.627719 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e388ead-5f0f-4b9a-b4d8-168749e46e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:32.628327 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e388ead-5f0f-4b9a-b4d8-168749e46e3d admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:32.628610 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e388ead-5f0f-4b9a-b4d8-168749e46e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:32.628884 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e388ead-5f0f-4b9a-b4d8-168749e46e3d admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:32.629512 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e388ead-5f0f-4b9a-b4d8-168749e46e3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:32.631091 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e388ead-5f0f-4b9a-b4d8-168749e46e3d admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:32.640819 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 978/1970] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:47:32 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:47:32.645476 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690cc38b-e103-48bb-9a7d-199edbbdf942 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:32.655667 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690cc38b-e103-48bb-9a7d-199edbbdf942 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:32.660949 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6c9c95-bba9-4b2e-aa2c-97f70a9e14de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:32.661662 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6c9c95-bba9-4b2e-aa2c-97f70a9e14de admin admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:32.661944 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6c9c95-bba9-4b2e-aa2c-97f70a9e14de admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:32.662218 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6c9c95-bba9-4b2e-aa2c-97f70a9e14de admin admin] PATH_INFO: `/v3/projects/79789607cba2404eae8e49fb4bc3a5e8/users/8ca90624010448a294c3651f752e209a/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:32.664164 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6c9c95-bba9-4b2e-aa2c-97f70a9e14de admin admin] RBAC: Authorizing `identity:create_grant(project_id=79789607cba2404eae8e49fb4bc3a5e8, user_id=8ca90624010448a294c3651f752e209a, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:32.667100 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6c9c95-bba9-4b2e-aa2c-97f70a9e14de admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:32.677968 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 993/1971] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:47:32 2022] PUT /identity/v3/projects/79789607cba2404eae8e49fb4bc3a5e8/users/8ca90624010448a294c3651f752e209a/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:47:32.687998 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dea67c0-c71c-4c5f-b2ce-92f0bfc60ce6 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:32.688283 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dea67c0-c71c-4c5f-b2ce-92f0bfc60ce6 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:32.688560 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dea67c0-c71c-4c5f-b2ce-92f0bfc60ce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:32.717845 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-1dea67c0-c71c-4c5f-b2ce-92f0bfc60ce6 None None] MFA Rules not processed for user `8ca90624010448a294c3651f752e209a`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:47:32.719051 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-1dea67c0-c71c-4c5f-b2ce-92f0bfc60ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:32.733545 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:32.800163 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 979/1972] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:47:32 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:47:32.814860 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6c9c95-bba9-4b2e-aa2c-97f70a9e14de admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:32.817246 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-7b6c9c95-bba9-4b2e-aa2c-97f70a9e14de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:32.843076 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:32.887275 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6c9c95-bba9-4b2e-aa2c-97f70a9e14de admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:32.891305 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e3fac9-f162-490a-865f-501be0eea8b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:32.891971 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3fac9-f162-490a-865f-501be0eea8b5 service cinder] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:32.892912 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3fac9-f162-490a-865f-501be0eea8b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:32.893251 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3fac9-f162-490a-865f-501be0eea8b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:32.969273 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e3fac9-f162-490a-865f-501be0eea8b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:32.971232 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e3fac9-f162-490a-865f-501be0eea8b5 service cinder] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:32.983085 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 994/1973] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:47:32 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:47:34.662636 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dea67c0-c71c-4c5f-b2ce-92f0bfc60ce6 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:34.676665 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dea67c0-c71c-4c5f-b2ce-92f0bfc60ce6 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:34.686470 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcf47dd-108d-4855-b0ab-f795b9a32454 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:34.689179 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcf47dd-108d-4855-b0ab-f795b9a32454 service cinder] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:34.689552 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcf47dd-108d-4855-b0ab-f795b9a32454 service cinder] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:34.689918 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcf47dd-108d-4855-b0ab-f795b9a32454 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:34.770938 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcf47dd-108d-4855-b0ab-f795b9a32454 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:34.772447 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcf47dd-108d-4855-b0ab-f795b9a32454 service cinder] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:34.784060 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 980/1974] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:47:34 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:47:35.327912 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e3fac9-f162-490a-865f-501be0eea8b5 service cinder] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:35.329521 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-c0e3fac9-f162-490a-865f-501be0eea8b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:35.400344 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e3fac9-f162-490a-865f-501be0eea8b5 service cinder] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:35.404919 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3b7063-63e0-497c-bb79-dea5dca984d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:35.406272 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3b7063-63e0-497c-bb79-dea5dca984d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:35.406272 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3b7063-63e0-497c-bb79-dea5dca984d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:35.406272 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3b7063-63e0-497c-bb79-dea5dca984d0 admin admin] PATH_INFO: `/v3/users/182869d74732426f8314d0eca33f7cde` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:35.407397 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3b7063-63e0-497c-bb79-dea5dca984d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=182869d74732426f8314d0eca33f7cde)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:35.410354 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3b7063-63e0-497c-bb79-dea5dca984d0 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:35.433775 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcf47dd-108d-4855-b0ab-f795b9a32454 service cinder] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:35.443316 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcf47dd-108d-4855-b0ab-f795b9a32454 service cinder] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:35.448207 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cdfd3d-0c44-480e-8ed0-9106d12a6232 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:35.449098 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cdfd3d-0c44-480e-8ed0-9106d12a6232 service cinder] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:35.449378 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cdfd3d-0c44-480e-8ed0-9106d12a6232 service cinder] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:35.449649 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cdfd3d-0c44-480e-8ed0-9106d12a6232 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:35.460872 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cdfd3d-0c44-480e-8ed0-9106d12a6232 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:35.462960 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cdfd3d-0c44-480e-8ed0-9106d12a6232 service cinder] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:35.475986 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 981/1975] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:47:35 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 10:47:35.482637 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-3e3b7063-63e0-497c-bb79-dea5dca984d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '182869d74732426f8314d0eca33f7cde'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:35.494916 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-3e3b7063-63e0-497c-bb79-dea5dca984d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '182869d74732426f8314d0eca33f7cde'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:35.495827 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-3e3b7063-63e0-497c-bb79-dea5dca984d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '182869d74732426f8314d0eca33f7cde'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:35.496771 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-3e3b7063-63e0-497c-bb79-dea5dca984d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '182869d74732426f8314d0eca33f7cde'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:35.511916 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-3e3b7063-63e0-497c-bb79-dea5dca984d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '182869d74732426f8314d0eca33f7cde'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:35.521643 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:35.535665 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 995/1976] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:47:35 2022] DELETE /identity/v3/users/182869d74732426f8314d0eca33f7cde => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:47:35.541027 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c8f4a7-2efc-4d34-a62e-7558ed6051d6 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:35.541443 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c8f4a7-2efc-4d34-a62e-7558ed6051d6 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:35.541845 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c8f4a7-2efc-4d34-a62e-7558ed6051d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:35.564725 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-a4c8f4a7-2efc-4d34-a62e-7558ed6051d6 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:47:35.566292 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-a4c8f4a7-2efc-4d34-a62e-7558ed6051d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:35.627321 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 982/1977] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:47:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:47:35.636878 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3b7063-63e0-497c-bb79-dea5dca984d0 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:35.638291 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-3e3b7063-63e0-497c-bb79-dea5dca984d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:35.696755 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3b7063-63e0-497c-bb79-dea5dca984d0 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:35.700479 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7456a88-ecb9-4c66-8de4-d2fb45483887 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:35.701091 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7456a88-ecb9-4c66-8de4-d2fb45483887 service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:35.701370 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7456a88-ecb9-4c66-8de4-d2fb45483887 service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:35.701649 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7456a88-ecb9-4c66-8de4-d2fb45483887 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:35.711540 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7456a88-ecb9-4c66-8de4-d2fb45483887 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:35.713340 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7456a88-ecb9-4c66-8de4-d2fb45483887 service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:35.725388 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 996/1978] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:47:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:47:36.157447 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c8f4a7-2efc-4d34-a62e-7558ed6051d6 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:36.160179 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-a4c8f4a7-2efc-4d34-a62e-7558ed6051d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:36.172532 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c8f4a7-2efc-4d34-a62e-7558ed6051d6 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:36.178402 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5535baf8-4982-423c-b061-c5a72b5e1ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:36.179239 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5535baf8-4982-423c-b061-c5a72b5e1ddb admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:36.179698 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5535baf8-4982-423c-b061-c5a72b5e1ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:36.179996 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5535baf8-4982-423c-b061-c5a72b5e1ddb admin admin] PATH_INFO: `/v3/projects/ee12a5a4bf2441b6ac4ac7d5220b3dec` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:36.181604 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5535baf8-4982-423c-b061-c5a72b5e1ddb admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee12a5a4bf2441b6ac4ac7d5220b3dec)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:36.183602 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5535baf8-4982-423c-b061-c5a72b5e1ddb admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:36.220773 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:36.233890 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-5535baf8-4982-423c-b061-c5a72b5e1ddb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee12a5a4bf2441b6ac4ac7d5220b3dec', 'request_id': 'req-5535baf8-4982-423c-b061-c5a72b5e1ddb'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:36.241218 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-5535baf8-4982-423c-b061-c5a72b5e1ddb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee12a5a4bf2441b6ac4ac7d5220b3dec', 'request_id': 'req-5535baf8-4982-423c-b061-c5a72b5e1ddb'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:36.248698 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-5535baf8-4982-423c-b061-c5a72b5e1ddb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee12a5a4bf2441b6ac4ac7d5220b3dec', 'request_id': 'req-5535baf8-4982-423c-b061-c5a72b5e1ddb'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:36.259626 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-5535baf8-4982-423c-b061-c5a72b5e1ddb admin admin] The token cache is being invalidate because project ee12a5a4bf2441b6ac4ac7d5220b3dec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90653) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:47:36.259913 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-5535baf8-4982-423c-b061-c5a72b5e1ddb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:36.260534 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-5535baf8-4982-423c-b061-c5a72b5e1ddb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:36.262048 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 983/1979] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:47:36 2022] DELETE /identity/v3/projects/ee12a5a4bf2441b6ac4ac7d5220b3dec => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 10:47:36.266080 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7456a88-ecb9-4c66-8de4-d2fb45483887 service neutron] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:36.276487 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7456a88-ecb9-4c66-8de4-d2fb45483887 service neutron] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:36.340853 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09d7153-376d-4758-bbcd-c08b11b064d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:36.341663 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09d7153-376d-4758-bbcd-c08b11b064d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:36.342353 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09d7153-376d-4758-bbcd-c08b11b064d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:36.342353 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09d7153-376d-4758-bbcd-c08b11b064d2 admin admin] PATH_INFO: `/v3/users/01e9fb97dd8548508745fc2c60fefee4` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:36.343510 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09d7153-376d-4758-bbcd-c08b11b064d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01e9fb97dd8548508745fc2c60fefee4)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:36.345655 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09d7153-376d-4758-bbcd-c08b11b064d2 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:36.393286 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-a09d7153-376d-4758-bbcd-c08b11b064d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01e9fb97dd8548508745fc2c60fefee4'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:36.401645 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-a09d7153-376d-4758-bbcd-c08b11b064d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01e9fb97dd8548508745fc2c60fefee4'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:36.402303 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-a09d7153-376d-4758-bbcd-c08b11b064d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01e9fb97dd8548508745fc2c60fefee4'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:36.402911 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-a09d7153-376d-4758-bbcd-c08b11b064d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01e9fb97dd8548508745fc2c60fefee4'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:36.415924 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-a09d7153-376d-4758-bbcd-c08b11b064d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01e9fb97dd8548508745fc2c60fefee4'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:36.428994 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 997/1980] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:47:36 2022] DELETE /identity/v3/users/01e9fb97dd8548508745fc2c60fefee4 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:47:36.946290 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5535baf8-4982-423c-b061-c5a72b5e1ddb admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:36.947826 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-5535baf8-4982-423c-b061-c5a72b5e1ddb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:36.962317 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5535baf8-4982-423c-b061-c5a72b5e1ddb admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:37.036509 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ff0405-5be3-4d64-b998-22ce1bde2bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:37.036954 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ff0405-5be3-4d64-b998-22ce1bde2bfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:37.037242 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ff0405-5be3-4d64-b998-22ce1bde2bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:37.037516 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ff0405-5be3-4d64-b998-22ce1bde2bfa admin admin] PATH_INFO: `/v3/projects/5f5f114f69a04fa786bbd18344d8a15d` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:37.039040 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ff0405-5be3-4d64-b998-22ce1bde2bfa admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f5f114f69a04fa786bbd18344d8a15d)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:37.041186 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ff0405-5be3-4d64-b998-22ce1bde2bfa admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:37.092214 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-28ff0405-5be3-4d64-b998-22ce1bde2bfa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f5f114f69a04fa786bbd18344d8a15d', 'request_id': 'req-28ff0405-5be3-4d64-b998-22ce1bde2bfa'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:37.098858 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-28ff0405-5be3-4d64-b998-22ce1bde2bfa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f5f114f69a04fa786bbd18344d8a15d', 'request_id': 'req-28ff0405-5be3-4d64-b998-22ce1bde2bfa'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:37.107721 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-28ff0405-5be3-4d64-b998-22ce1bde2bfa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f5f114f69a04fa786bbd18344d8a15d', 'request_id': 'req-28ff0405-5be3-4d64-b998-22ce1bde2bfa'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:37.119403 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-28ff0405-5be3-4d64-b998-22ce1bde2bfa admin admin] The token cache is being invalidate because project 5f5f114f69a04fa786bbd18344d8a15d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90653) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:47:37.119724 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-28ff0405-5be3-4d64-b998-22ce1bde2bfa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:37.120368 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-28ff0405-5be3-4d64-b998-22ce1bde2bfa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:37.121940 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 984/1981] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:47:36 2022] DELETE /identity/v3/projects/5f5f114f69a04fa786bbd18344d8a15d => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:47:37.128729 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21a3d6b-9fb2-48ac-bd32-698aa63b8ec2 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:37.129012 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21a3d6b-9fb2-48ac-bd32-698aa63b8ec2 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:37.129286 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21a3d6b-9fb2-48ac-bd32-698aa63b8ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:37.149569 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-f21a3d6b-9fb2-48ac-bd32-698aa63b8ec2 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:47:37.150786 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-f21a3d6b-9fb2-48ac-bd32-698aa63b8ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:37.162617 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:37.251050 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 998/1982] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:47:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 10:47:37.261403 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d31950-6e2f-4dc7-ae39-d0f8ffd4d9b2 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:37.261880 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d31950-6e2f-4dc7-ae39-d0f8ffd4d9b2 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:37.261970 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d31950-6e2f-4dc7-ae39-d0f8ffd4d9b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:37.283097 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-80d31950-6e2f-4dc7-ae39-d0f8ffd4d9b2 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:47:37.284893 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-80d31950-6e2f-4dc7-ae39-d0f8ffd4d9b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:37.292673 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 985/1983] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:47:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:47:37.296844 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21a3d6b-9fb2-48ac-bd32-698aa63b8ec2 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:37.307410 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21a3d6b-9fb2-48ac-bd32-698aa63b8ec2 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:37.312321 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e5ccf4-a4e8-4ecc-b382-3208647a5688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:37.312928 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e5ccf4-a4e8-4ecc-b382-3208647a5688 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:37.313205 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e5ccf4-a4e8-4ecc-b382-3208647a5688 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:37.313897 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e5ccf4-a4e8-4ecc-b382-3208647a5688 admin admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:37.314670 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e5ccf4-a4e8-4ecc-b382-3208647a5688 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:47:37.315611 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e5ccf4-a4e8-4ecc-b382-3208647a5688 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:37.316978 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e5ccf4-a4e8-4ecc-b382-3208647a5688 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:37.328934 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 999/1984] 10.222.0.15 () {58 vars in 1153 bytes} [Sun Sep 11 10:47:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:47:37.333146 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d31950-6e2f-4dc7-ae39-d0f8ffd4d9b2 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:37.342839 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d31950-6e2f-4dc7-ae39-d0f8ffd4d9b2 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:37.347882 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c19a362-9580-4993-8850-10df0041526c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:37.348483 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c19a362-9580-4993-8850-10df0041526c admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:37.348758 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c19a362-9580-4993-8850-10df0041526c admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:37.349024 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c19a362-9580-4993-8850-10df0041526c admin admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:37.349654 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c19a362-9580-4993-8850-10df0041526c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:37.351736 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c19a362-9580-4993-8850-10df0041526c admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:37.383741 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 986/1985] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:47:37 2022] POST /identity/v3/projects => generated 361 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:47:37.384938 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e5ccf4-a4e8-4ecc-b382-3208647a5688 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:37.395741 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e5ccf4-a4e8-4ecc-b382-3208647a5688 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:37.456654 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32015989-5bb5-48cb-9df3-c4c27e69345c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:37.457280 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32015989-5bb5-48cb-9df3-c4c27e69345c admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:37.457590 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32015989-5bb5-48cb-9df3-c4c27e69345c admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:37.457895 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32015989-5bb5-48cb-9df3-c4c27e69345c admin admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:37.458494 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32015989-5bb5-48cb-9df3-c4c27e69345c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:37.460544 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32015989-5bb5-48cb-9df3-c4c27e69345c admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:37.487379 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1000/1986] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:47:37 2022] POST /identity/v3/users => generated 281 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 10:47:37.488366 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c19a362-9580-4993-8850-10df0041526c admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:37.499045 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c19a362-9580-4993-8850-10df0041526c admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:37.504895 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481ea321-9d0c-44c1-ac27-21a92318eec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:37.505511 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481ea321-9d0c-44c1-ac27-21a92318eec1 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:37.505790 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481ea321-9d0c-44c1-ac27-21a92318eec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:37.506088 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481ea321-9d0c-44c1-ac27-21a92318eec1 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:37.506754 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481ea321-9d0c-44c1-ac27-21a92318eec1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:37.508416 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481ea321-9d0c-44c1-ac27-21a92318eec1 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:37.521659 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 987/1987] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:47:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:47:37.523490 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32015989-5bb5-48cb-9df3-c4c27e69345c admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:37.533872 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32015989-5bb5-48cb-9df3-c4c27e69345c admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:37.539198 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e20cbb-aed2-4539-8b2b-ba5ac45f9403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:37.539845 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e20cbb-aed2-4539-8b2b-ba5ac45f9403 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:37.540125 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e20cbb-aed2-4539-8b2b-ba5ac45f9403 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:37.540395 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e20cbb-aed2-4539-8b2b-ba5ac45f9403 admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:37.541077 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e20cbb-aed2-4539-8b2b-ba5ac45f9403 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:37.542616 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e20cbb-aed2-4539-8b2b-ba5ac45f9403 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:37.554268 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1001/1988] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:47:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:47:37.558474 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481ea321-9d0c-44c1-ac27-21a92318eec1 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:37.569000 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481ea321-9d0c-44c1-ac27-21a92318eec1 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:37.573673 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5f48e7-64ce-4f41-9dcd-fe07a27834f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:37.574348 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f48e7-64ce-4f41-9dcd-fe07a27834f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:37.574633 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f48e7-64ce-4f41-9dcd-fe07a27834f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:37.574904 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f48e7-64ce-4f41-9dcd-fe07a27834f4 admin admin] PATH_INFO: `/v3/projects/6d71f5dba68c4cb3870c0ca333f0f23e/users/f88c2886d9d741bc8d4302993a3ca320/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:37.583906 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:37.592201 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5f48e7-64ce-4f41-9dcd-fe07a27834f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d71f5dba68c4cb3870c0ca333f0f23e, user_id=f88c2886d9d741bc8d4302993a3ca320, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:37.595513 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5f48e7-64ce-4f41-9dcd-fe07a27834f4 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:37.610894 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 988/1989] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:47:37 2022] PUT /identity/v3/projects/6d71f5dba68c4cb3870c0ca333f0f23e/users/f88c2886d9d741bc8d4302993a3ca320/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:47:37.622135 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bb20c2-6bf8-4848-a1cc-5df5acf3bdfa None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:37.622421 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bb20c2-6bf8-4848-a1cc-5df5acf3bdfa None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:37.622693 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bb20c2-6bf8-4848-a1cc-5df5acf3bdfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:37.651344 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-84bb20c2-6bf8-4848-a1cc-5df5acf3bdfa None None] MFA Rules not processed for user `f88c2886d9d741bc8d4302993a3ca320`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:47:37.652565 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-84bb20c2-6bf8-4848-a1cc-5df5acf3bdfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:37.719349 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1002/1990] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:47:37 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:47:37.754986 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5f48e7-64ce-4f41-9dcd-fe07a27834f4 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:37.756722 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-7b5f48e7-64ce-4f41-9dcd-fe07a27834f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:37.814243 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5f48e7-64ce-4f41-9dcd-fe07a27834f4 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:37.817786 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e54c0ea-dbb5-48ec-9f7c-986df210ad31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:37.818384 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e54c0ea-dbb5-48ec-9f7c-986df210ad31 service cinder] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:37.818663 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e54c0ea-dbb5-48ec-9f7c-986df210ad31 service cinder] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:37.818935 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e54c0ea-dbb5-48ec-9f7c-986df210ad31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:37.827014 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e54c0ea-dbb5-48ec-9f7c-986df210ad31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:37.828474 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e54c0ea-dbb5-48ec-9f7c-986df210ad31 service cinder] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:37.837654 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 989/1991] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:47:37 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:47:37.853392 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bb20c2-6bf8-4848-a1cc-5df5acf3bdfa None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:37.866180 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:37.920582 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bb20c2-6bf8-4848-a1cc-5df5acf3bdfa None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:37.925763 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53c4d54-2548-44fe-98d6-473303690932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:37.926491 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c4d54-2548-44fe-98d6-473303690932 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:37.926770 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c4d54-2548-44fe-98d6-473303690932 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:37.927099 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c4d54-2548-44fe-98d6-473303690932 admin admin] PATH_INFO: `/v3/users/f88c2886d9d741bc8d4302993a3ca320` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:37.928343 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53c4d54-2548-44fe-98d6-473303690932 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f88c2886d9d741bc8d4302993a3ca320)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:37.930314 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53c4d54-2548-44fe-98d6-473303690932 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:37.981963 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-e53c4d54-2548-44fe-98d6-473303690932 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f88c2886d9d741bc8d4302993a3ca320'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:37.990344 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-e53c4d54-2548-44fe-98d6-473303690932 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f88c2886d9d741bc8d4302993a3ca320'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:37.991002 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-e53c4d54-2548-44fe-98d6-473303690932 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f88c2886d9d741bc8d4302993a3ca320'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:37.991652 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-e53c4d54-2548-44fe-98d6-473303690932 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f88c2886d9d741bc8d4302993a3ca320'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:38.004636 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-e53c4d54-2548-44fe-98d6-473303690932 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f88c2886d9d741bc8d4302993a3ca320'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:38.017806 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1003/1992] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:47:37 2022] DELETE /identity/v3/users/f88c2886d9d741bc8d4302993a3ca320 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:47:38.022215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5081835-c7f7-4108-8155-b0c32d1ca844 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:38.022500 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5081835-c7f7-4108-8155-b0c32d1ca844 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:38.022775 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5081835-c7f7-4108-8155-b0c32d1ca844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:38.041629 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-a5081835-c7f7-4108-8155-b0c32d1ca844 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:47:38.042820 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-a5081835-c7f7-4108-8155-b0c32d1ca844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:38.118299 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 990/1993] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:47:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:47:38.128181 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53c4d54-2548-44fe-98d6-473303690932 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:38.129565 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-e53c4d54-2548-44fe-98d6-473303690932 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:38.188380 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53c4d54-2548-44fe-98d6-473303690932 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:38.191998 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca35b2a3-b5e5-4557-8b67-e66c72ac85bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:38.192604 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca35b2a3-b5e5-4557-8b67-e66c72ac85bc service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:38.192882 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca35b2a3-b5e5-4557-8b67-e66c72ac85bc service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:38.193152 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca35b2a3-b5e5-4557-8b67-e66c72ac85bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:38.202085 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca35b2a3-b5e5-4557-8b67-e66c72ac85bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:38.203520 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca35b2a3-b5e5-4557-8b67-e66c72ac85bc service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:38.212837 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1004/1994] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:47:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:47:38.643606 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5081835-c7f7-4108-8155-b0c32d1ca844 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:38.645033 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-a5081835-c7f7-4108-8155-b0c32d1ca844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:38.657141 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5081835-c7f7-4108-8155-b0c32d1ca844 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:38.663998 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc89d09-f5b9-49a6-ab48-22d510867066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:38.664650 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc89d09-f5b9-49a6-ab48-22d510867066 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:38.664934 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc89d09-f5b9-49a6-ab48-22d510867066 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:38.665208 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc89d09-f5b9-49a6-ab48-22d510867066 admin admin] PATH_INFO: `/v3/projects/6d71f5dba68c4cb3870c0ca333f0f23e` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:38.666341 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc89d09-f5b9-49a6-ab48-22d510867066 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d71f5dba68c4cb3870c0ca333f0f23e)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:38.668304 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc89d09-f5b9-49a6-ab48-22d510867066 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:38.719728 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-7cc89d09-f5b9-49a6-ab48-22d510867066 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d71f5dba68c4cb3870c0ca333f0f23e', 'request_id': 'req-7cc89d09-f5b9-49a6-ab48-22d510867066'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:38.726946 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-7cc89d09-f5b9-49a6-ab48-22d510867066 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d71f5dba68c4cb3870c0ca333f0f23e', 'request_id': 'req-7cc89d09-f5b9-49a6-ab48-22d510867066'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:38.738558 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-7cc89d09-f5b9-49a6-ab48-22d510867066 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d71f5dba68c4cb3870c0ca333f0f23e', 'request_id': 'req-7cc89d09-f5b9-49a6-ab48-22d510867066'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:38.755182 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-7cc89d09-f5b9-49a6-ab48-22d510867066 admin admin] The token cache is being invalidate because project 6d71f5dba68c4cb3870c0ca333f0f23e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90653) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:47:38.755742 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-7cc89d09-f5b9-49a6-ab48-22d510867066 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:38.757791 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-7cc89d09-f5b9-49a6-ab48-22d510867066 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:38.759403 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 991/1995] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:47:38 2022] DELETE /identity/v3/projects/6d71f5dba68c4cb3870c0ca333f0f23e => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:47:38.764143 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d4fa1e-c6f1-4e80-a9c6-a47e1a019f19 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:38.764644 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d4fa1e-c6f1-4e80-a9c6-a47e1a019f19 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:38.764823 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d4fa1e-c6f1-4e80-a9c6-a47e1a019f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:38.787893 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-61d4fa1e-c6f1-4e80-a9c6-a47e1a019f19 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:47:38.789204 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-61d4fa1e-c6f1-4e80-a9c6-a47e1a019f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:38.826725 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:38.865499 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1005/1996] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:47:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 10:47:38.873245 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0071927b-91e7-4804-a814-9a3a3f73eabc None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:38.873539 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0071927b-91e7-4804-a814-9a3a3f73eabc None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:38.873818 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0071927b-91e7-4804-a814-9a3a3f73eabc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:38.895495 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-0071927b-91e7-4804-a814-9a3a3f73eabc None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:47:38.897609 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-0071927b-91e7-4804-a814-9a3a3f73eabc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:38.906774 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 992/1997] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:47:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:47:38.911180 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d4fa1e-c6f1-4e80-a9c6-a47e1a019f19 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:38.920967 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d4fa1e-c6f1-4e80-a9c6-a47e1a019f19 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:38.925613 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16193f32-ee9f-4773-8215-e6b44c3e31f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:38.926215 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16193f32-ee9f-4773-8215-e6b44c3e31f9 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:38.926499 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16193f32-ee9f-4773-8215-e6b44c3e31f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:38.926776 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16193f32-ee9f-4773-8215-e6b44c3e31f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:38.927442 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16193f32-ee9f-4773-8215-e6b44c3e31f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:47:38.927757 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16193f32-ee9f-4773-8215-e6b44c3e31f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:38.929346 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16193f32-ee9f-4773-8215-e6b44c3e31f9 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:38.941755 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1006/1998] 10.222.0.15 () {58 vars in 1153 bytes} [Sun Sep 11 10:47:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:47:38.946141 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0071927b-91e7-4804-a814-9a3a3f73eabc None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:38.956057 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0071927b-91e7-4804-a814-9a3a3f73eabc None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:38.960772 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743d2629-4e47-459c-9955-78d8cebc8e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:38.961365 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d2629-4e47-459c-9955-78d8cebc8e7c admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:38.961648 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d2629-4e47-459c-9955-78d8cebc8e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:38.961922 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d2629-4e47-459c-9955-78d8cebc8e7c admin admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:38.962542 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743d2629-4e47-459c-9955-78d8cebc8e7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:38.964477 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743d2629-4e47-459c-9955-78d8cebc8e7c admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:38.968886 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:38.995024 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 993/1999] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:47:38 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 10:47:38.997090 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16193f32-ee9f-4773-8215-e6b44c3e31f9 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:39.007391 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16193f32-ee9f-4773-8215-e6b44c3e31f9 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:39.063026 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a44734-bce9-405a-b0a2-43097127ef51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:39.063656 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a44734-bce9-405a-b0a2-43097127ef51 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:39.064481 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a44734-bce9-405a-b0a2-43097127ef51 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:39.064740 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a44734-bce9-405a-b0a2-43097127ef51 admin admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:39.065377 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a44734-bce9-405a-b0a2-43097127ef51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:39.067281 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a44734-bce9-405a-b0a2-43097127ef51 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:39.088565 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1007/2000] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:47:38 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:47:39.092705 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743d2629-4e47-459c-9955-78d8cebc8e7c admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:39.103227 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743d2629-4e47-459c-9955-78d8cebc8e7c admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:39.108230 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebc1ca9-584c-4f74-9ea0-e2464a5b10d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:39.108824 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebc1ca9-584c-4f74-9ea0-e2464a5b10d9 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:39.109107 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebc1ca9-584c-4f74-9ea0-e2464a5b10d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:39.109383 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebc1ca9-584c-4f74-9ea0-e2464a5b10d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:39.110098 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebc1ca9-584c-4f74-9ea0-e2464a5b10d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:39.117220 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebc1ca9-584c-4f74-9ea0-e2464a5b10d9 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:39.123224 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 994/2001] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:47:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:47:39.128040 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a44734-bce9-405a-b0a2-43097127ef51 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:39.138375 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a44734-bce9-405a-b0a2-43097127ef51 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:39.144083 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b20c96e-2d46-449a-ae1e-25ef58ff121a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:39.144696 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b20c96e-2d46-449a-ae1e-25ef58ff121a admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:39.144975 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b20c96e-2d46-449a-ae1e-25ef58ff121a admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:39.145247 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b20c96e-2d46-449a-ae1e-25ef58ff121a admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:39.145941 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b20c96e-2d46-449a-ae1e-25ef58ff121a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:39.147507 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b20c96e-2d46-449a-ae1e-25ef58ff121a admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:39.158582 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1008/2002] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:47:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:47:39.162794 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebc1ca9-584c-4f74-9ea0-e2464a5b10d9 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:39.172943 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebc1ca9-584c-4f74-9ea0-e2464a5b10d9 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:39.177789 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd551a7e-53c5-47e0-9016-4b9d01cede29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:39.178477 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd551a7e-53c5-47e0-9016-4b9d01cede29 admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:39.178761 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd551a7e-53c5-47e0-9016-4b9d01cede29 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:39.179039 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd551a7e-53c5-47e0-9016-4b9d01cede29 admin admin] PATH_INFO: `/v3/projects/423d512e5a254b59ac007f2bb2d79dc8/users/925516134555460c9025b8dae680eeb8/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:39.194890 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd551a7e-53c5-47e0-9016-4b9d01cede29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=423d512e5a254b59ac007f2bb2d79dc8, user_id=925516134555460c9025b8dae680eeb8, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:39.197714 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd551a7e-53c5-47e0-9016-4b9d01cede29 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:39.212830 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 995/2003] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:47:39 2022] PUT /identity/v3/projects/423d512e5a254b59ac007f2bb2d79dc8/users/925516134555460c9025b8dae680eeb8/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 10:47:39.223954 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb41c64-5e4d-4318-addf-e0f9b37ed936 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:39.224239 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb41c64-5e4d-4318-addf-e0f9b37ed936 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:39.224514 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb41c64-5e4d-4318-addf-e0f9b37ed936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:39.254891 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-afb41c64-5e4d-4318-addf-e0f9b37ed936 None None] MFA Rules not processed for user `925516134555460c9025b8dae680eeb8`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:47:39.256153 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-afb41c64-5e4d-4318-addf-e0f9b37ed936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:39.328994 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1009/2004] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:47:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:47:39.340664 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd551a7e-53c5-47e0-9016-4b9d01cede29 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:39.342353 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-dd551a7e-53c5-47e0-9016-4b9d01cede29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:39.415991 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd551a7e-53c5-47e0-9016-4b9d01cede29 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:39.419693 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2576845f-4942-4061-8530-b92fa0e0effd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:39.420304 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576845f-4942-4061-8530-b92fa0e0effd service cinder] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:39.420638 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576845f-4942-4061-8530-b92fa0e0effd service cinder] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:39.420853 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576845f-4942-4061-8530-b92fa0e0effd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:39.431786 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2576845f-4942-4061-8530-b92fa0e0effd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:39.433325 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2576845f-4942-4061-8530-b92fa0e0effd service cinder] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:39.443783 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 996/2005] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:47:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:47:49.326035 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=84,keystone:INSERT=3 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:49.438731 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=47,keystone:INSERT=2 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:50.175466 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb41c64-5e4d-4318-addf-e0f9b37ed936 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:50.177500 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-afb41c64-5e4d-4318-addf-e0f9b37ed936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:50.232882 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2576845f-4942-4061-8530-b92fa0e0effd service cinder] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:50.270754 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb41c64-5e4d-4318-addf-e0f9b37ed936 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:50.276143 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a90c151-d49f-4377-a5f4-4aff653bd344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:50.276816 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a90c151-d49f-4377-a5f4-4aff653bd344 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:50.277168 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a90c151-d49f-4377-a5f4-4aff653bd344 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:50.277378 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a90c151-d49f-4377-a5f4-4aff653bd344 admin admin] PATH_INFO: `/v3/users/d02554a02da54173bd4dd3200feaf392` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:50.278966 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a90c151-d49f-4377-a5f4-4aff653bd344 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d02554a02da54173bd4dd3200feaf392)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:50.280939 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a90c151-d49f-4377-a5f4-4aff653bd344 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:50.336178 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2576845f-4942-4061-8530-b92fa0e0effd service cinder] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:50.340688 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5382abf-8ebe-4c67-839c-d4926d9bac4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:50.341469 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5382abf-8ebe-4c67-839c-d4926d9bac4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:50.341760 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5382abf-8ebe-4c67-839c-d4926d9bac4b admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:50.342036 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5382abf-8ebe-4c67-839c-d4926d9bac4b admin admin] PATH_INFO: `/v3/users/925516134555460c9025b8dae680eeb8` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:50.342985 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-4a90c151-d49f-4377-a5f4-4aff653bd344 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd02554a02da54173bd4dd3200feaf392'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:50.343471 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5382abf-8ebe-4c67-839c-d4926d9bac4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=925516134555460c9025b8dae680eeb8)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:50.345914 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5382abf-8ebe-4c67-839c-d4926d9bac4b admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:50.355995 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-4a90c151-d49f-4377-a5f4-4aff653bd344 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd02554a02da54173bd4dd3200feaf392'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:50.357439 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-4a90c151-d49f-4377-a5f4-4aff653bd344 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd02554a02da54173bd4dd3200feaf392'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:50.358824 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-4a90c151-d49f-4377-a5f4-4aff653bd344 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd02554a02da54173bd4dd3200feaf392'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:50.391643 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-4a90c151-d49f-4377-a5f4-4aff653bd344 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd02554a02da54173bd4dd3200feaf392'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:50.404948 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-c5382abf-8ebe-4c67-839c-d4926d9bac4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '925516134555460c9025b8dae680eeb8'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:50.406166 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1010/2006] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:47:50 2022] DELETE /identity/v3/users/d02554a02da54173bd4dd3200feaf392 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:47:50.411109 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf146ec-15df-463f-85e5-c2a0f4f435da None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:50.411747 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf146ec-15df-463f-85e5-c2a0f4f435da None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:50.412251 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf146ec-15df-463f-85e5-c2a0f4f435da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:50.416041 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-c5382abf-8ebe-4c67-839c-d4926d9bac4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '925516134555460c9025b8dae680eeb8'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:50.416733 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-c5382abf-8ebe-4c67-839c-d4926d9bac4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '925516134555460c9025b8dae680eeb8'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:50.417547 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-c5382abf-8ebe-4c67-839c-d4926d9bac4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '925516134555460c9025b8dae680eeb8'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:50.425383 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-c5382abf-8ebe-4c67-839c-d4926d9bac4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '925516134555460c9025b8dae680eeb8'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:50.443775 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 997/2007] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:47:50 2022] DELETE /identity/v3/users/925516134555460c9025b8dae680eeb8 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 10:47:50.447122 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-7cf146ec-15df-463f-85e5-c2a0f4f435da None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:47:50.447122 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-7cf146ec-15df-463f-85e5-c2a0f4f435da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:50.448002 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e5399a-27ae-492f-aaa9-2e7c511b62a3 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:50.448287 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e5399a-27ae-492f-aaa9-2e7c511b62a3 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:50.448561 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e5399a-27ae-492f-aaa9-2e7c511b62a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:50.475630 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-e8e5399a-27ae-492f-aaa9-2e7c511b62a3 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:47:50.476850 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-e8e5399a-27ae-492f-aaa9-2e7c511b62a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:50.520166 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1011/2008] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:47:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 10:47:50.530457 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf146ec-15df-463f-85e5-c2a0f4f435da None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:50.531919 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-7cf146ec-15df-463f-85e5-c2a0f4f435da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:50.541902 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 998/2009] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:47:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 10:47:50.552023 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e5399a-27ae-492f-aaa9-2e7c511b62a3 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:50.553383 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-e8e5399a-27ae-492f-aaa9-2e7c511b62a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:50.556101 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:50.575724 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:50.602637 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf146ec-15df-463f-85e5-c2a0f4f435da None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:50.606300 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a0d267-f85f-4ac2-9410-287bb833dedf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:50.607172 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a0d267-f85f-4ac2-9410-287bb833dedf service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:50.607475 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a0d267-f85f-4ac2-9410-287bb833dedf service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:50.607743 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a0d267-f85f-4ac2-9410-287bb833dedf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:50.617263 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a0d267-f85f-4ac2-9410-287bb833dedf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:50.618722 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a0d267-f85f-4ac2-9410-287bb833dedf service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:50.619059 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e5399a-27ae-492f-aaa9-2e7c511b62a3 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:50.622712 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48df7ea-6c0f-4583-a634-983ae77cb097 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:50.623356 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48df7ea-6c0f-4583-a634-983ae77cb097 service neutron] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:50.623660 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48df7ea-6c0f-4583-a634-983ae77cb097 service neutron] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:50.623932 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48df7ea-6c0f-4583-a634-983ae77cb097 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:50.628234 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1012/2010] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:47:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 10:47:50.633609 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48df7ea-6c0f-4583-a634-983ae77cb097 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:50.634883 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48df7ea-6c0f-4583-a634-983ae77cb097 service neutron] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:50.646375 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 999/2011] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:47:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 10:47:51.706511 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a0d267-f85f-4ac2-9410-287bb833dedf service neutron] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:51.709096 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-24a0d267-f85f-4ac2-9410-287bb833dedf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:51.733660 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a0d267-f85f-4ac2-9410-287bb833dedf service neutron] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:51.742903 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62353c60-85b1-43e7-8e1c-2dc6fa520bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:51.744449 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62353c60-85b1-43e7-8e1c-2dc6fa520bf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:51.745374 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62353c60-85b1-43e7-8e1c-2dc6fa520bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:51.746431 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62353c60-85b1-43e7-8e1c-2dc6fa520bf6 admin admin] PATH_INFO: `/v3/projects/059ded7d632a4e0e91e74d5cdde54725` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:51.749140 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62353c60-85b1-43e7-8e1c-2dc6fa520bf6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=059ded7d632a4e0e91e74d5cdde54725)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:51.753802 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62353c60-85b1-43e7-8e1c-2dc6fa520bf6 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:51.768971 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48df7ea-6c0f-4583-a634-983ae77cb097 service neutron] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:51.773842 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-d48df7ea-6c0f-4583-a634-983ae77cb097 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:51.805919 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48df7ea-6c0f-4583-a634-983ae77cb097 service neutron] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:51.826526 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36192ffa-38d0-4136-925d-71361be798b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:51.829106 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36192ffa-38d0-4136-925d-71361be798b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:51.831127 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36192ffa-38d0-4136-925d-71361be798b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:51.831812 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36192ffa-38d0-4136-925d-71361be798b8 admin admin] PATH_INFO: `/v3/projects/423d512e5a254b59ac007f2bb2d79dc8` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:51.834936 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36192ffa-38d0-4136-925d-71361be798b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=423d512e5a254b59ac007f2bb2d79dc8)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:51.840083 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36192ffa-38d0-4136-925d-71361be798b8 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:51.944647 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-62353c60-85b1-43e7-8e1c-2dc6fa520bf6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '059ded7d632a4e0e91e74d5cdde54725', 'request_id': 'req-62353c60-85b1-43e7-8e1c-2dc6fa520bf6'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:51.956245 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-62353c60-85b1-43e7-8e1c-2dc6fa520bf6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '059ded7d632a4e0e91e74d5cdde54725', 'request_id': 'req-62353c60-85b1-43e7-8e1c-2dc6fa520bf6'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:51.964026 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-62353c60-85b1-43e7-8e1c-2dc6fa520bf6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '059ded7d632a4e0e91e74d5cdde54725', 'request_id': 'req-62353c60-85b1-43e7-8e1c-2dc6fa520bf6'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:51.978386 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-36192ffa-38d0-4136-925d-71361be798b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '423d512e5a254b59ac007f2bb2d79dc8', 'request_id': 'req-36192ffa-38d0-4136-925d-71361be798b8'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:51.987979 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-36192ffa-38d0-4136-925d-71361be798b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '423d512e5a254b59ac007f2bb2d79dc8', 'request_id': 'req-36192ffa-38d0-4136-925d-71361be798b8'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:51.989486 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-62353c60-85b1-43e7-8e1c-2dc6fa520bf6 admin admin] The token cache is being invalidate because project 059ded7d632a4e0e91e74d5cdde54725 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:47:51.989486 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-62353c60-85b1-43e7-8e1c-2dc6fa520bf6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:51.990151 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-62353c60-85b1-43e7-8e1c-2dc6fa520bf6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:51.992310 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1013/2012] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:47:51 2022] DELETE /identity/v3/projects/059ded7d632a4e0e91e74d5cdde54725 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:47:51.996306 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62353c60-85b1-43e7-8e1c-2dc6fa520bf6 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:51.999300 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-62353c60-85b1-43e7-8e1c-2dc6fa520bf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:52.000562 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-36192ffa-38d0-4136-925d-71361be798b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '423d512e5a254b59ac007f2bb2d79dc8', 'request_id': 'req-36192ffa-38d0-4136-925d-71361be798b8'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:52.029910 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-36192ffa-38d0-4136-925d-71361be798b8 admin admin] The token cache is being invalidate because project 423d512e5a254b59ac007f2bb2d79dc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90653) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:47:52.030217 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-36192ffa-38d0-4136-925d-71361be798b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:52.031785 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-36192ffa-38d0-4136-925d-71361be798b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:52.035916 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1000/2013] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:47:51 2022] DELETE /identity/v3/projects/423d512e5a254b59ac007f2bb2d79dc8 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:47:52.045776 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e166fcc5-5faa-4bf4-ad5f-f168231a7d57 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:52.046170 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e166fcc5-5faa-4bf4-ad5f-f168231a7d57 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:52.046555 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e166fcc5-5faa-4bf4-ad5f-f168231a7d57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:52.113296 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-e166fcc5-5faa-4bf4-ad5f-f168231a7d57 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:47:52.115035 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-e166fcc5-5faa-4bf4-ad5f-f168231a7d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:52.130019 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62353c60-85b1-43e7-8e1c-2dc6fa520bf6 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:52.194237 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1001/2014] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:47:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:47:52.201545 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7b19fe-30c5-4182-ae28-0cf373592ff5 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:52.201791 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7b19fe-30c5-4182-ae28-0cf373592ff5 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:52.202067 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7b19fe-30c5-4182-ae28-0cf373592ff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:52.215747 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:52.225976 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-5c7b19fe-30c5-4182-ae28-0cf373592ff5 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:47:52.228743 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-5c7b19fe-30c5-4182-ae28-0cf373592ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:52.230359 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7865bcd3-21a6-4888-9dfa-d8a66137d714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:52.230953 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7865bcd3-21a6-4888-9dfa-d8a66137d714 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:52.231400 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7865bcd3-21a6-4888-9dfa-d8a66137d714 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:52.231708 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7865bcd3-21a6-4888-9dfa-d8a66137d714 admin admin] PATH_INFO: `/v3/users/2dc88790380e4c3e84f8f5f0db3400d5` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:52.233483 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7865bcd3-21a6-4888-9dfa-d8a66137d714 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2dc88790380e4c3e84f8f5f0db3400d5)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:52.235869 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7865bcd3-21a6-4888-9dfa-d8a66137d714 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:52.243071 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1002/2015] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:47:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 10:47:52.247533 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7b19fe-30c5-4182-ae28-0cf373592ff5 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:52.258190 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7b19fe-30c5-4182-ae28-0cf373592ff5 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:52.262941 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a2463b-2da5-4243-ac9a-0a164d0b1ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:52.264282 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a2463b-2da5-4243-ac9a-0a164d0b1ce0 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:52.264799 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a2463b-2da5-4243-ac9a-0a164d0b1ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:52.265071 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a2463b-2da5-4243-ac9a-0a164d0b1ce0 admin admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:52.265860 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a2463b-2da5-4243-ac9a-0a164d0b1ce0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:47:52.267727 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a2463b-2da5-4243-ac9a-0a164d0b1ce0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:52.270648 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a2463b-2da5-4243-ac9a-0a164d0b1ce0 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:52.292303 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1003/2016] 10.222.0.15 () {58 vars in 1153 bytes} [Sun Sep 11 10:47:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:47:52.301022 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-7865bcd3-21a6-4888-9dfa-d8a66137d714 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2dc88790380e4c3e84f8f5f0db3400d5'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:52.301820 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a2463b-2da5-4243-ac9a-0a164d0b1ce0 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:52.314706 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a2463b-2da5-4243-ac9a-0a164d0b1ce0 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:52.316459 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-7865bcd3-21a6-4888-9dfa-d8a66137d714 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2dc88790380e4c3e84f8f5f0db3400d5'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:52.317122 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-7865bcd3-21a6-4888-9dfa-d8a66137d714 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2dc88790380e4c3e84f8f5f0db3400d5'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:52.317844 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-7865bcd3-21a6-4888-9dfa-d8a66137d714 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2dc88790380e4c3e84f8f5f0db3400d5'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:52.324540 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cff975-dae1-458f-a3dd-b96d821e4663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:52.325330 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cff975-dae1-458f-a3dd-b96d821e4663 admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:52.325963 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cff975-dae1-458f-a3dd-b96d821e4663 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:52.326421 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cff975-dae1-458f-a3dd-b96d821e4663 admin admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:52.327693 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cff975-dae1-458f-a3dd-b96d821e4663 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:52.330151 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cff975-dae1-458f-a3dd-b96d821e4663 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:52.331275 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-7865bcd3-21a6-4888-9dfa-d8a66137d714 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2dc88790380e4c3e84f8f5f0db3400d5'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:52.343834 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1014/2017] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:47:51 2022] DELETE /identity/v3/users/2dc88790380e4c3e84f8f5f0db3400d5 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 10:47:52.359465 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1004/2018] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:47:52 2022] POST /identity/v3/projects => generated 369 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:47:52.363241 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7865bcd3-21a6-4888-9dfa-d8a66137d714 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:52.364675 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-7865bcd3-21a6-4888-9dfa-d8a66137d714 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:52.433271 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7865bcd3-21a6-4888-9dfa-d8a66137d714 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:52.493054 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9c5403-997f-4a97-9289-83f2d8d85332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:52.493811 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9c5403-997f-4a97-9289-83f2d8d85332 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:52.494007 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9c5403-997f-4a97-9289-83f2d8d85332 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:52.494286 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9c5403-997f-4a97-9289-83f2d8d85332 admin admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:52.494923 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9c5403-997f-4a97-9289-83f2d8d85332 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:52.496920 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9c5403-997f-4a97-9289-83f2d8d85332 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:52.522906 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1015/2019] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:47:52 2022] POST /identity/v3/users => generated 285 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:47:52.524740 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cff975-dae1-458f-a3dd-b96d821e4663 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:52.532745 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:52.537233 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cff975-dae1-458f-a3dd-b96d821e4663 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:52.543501 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a2a256-fc34-4e03-bec6-f0577abd9df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:52.544401 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a2a256-fc34-4e03-bec6-f0577abd9df4 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:52.544856 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a2a256-fc34-4e03-bec6-f0577abd9df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:52.545359 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a2a256-fc34-4e03-bec6-f0577abd9df4 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:52.546200 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a2a256-fc34-4e03-bec6-f0577abd9df4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:52.548359 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a2a256-fc34-4e03-bec6-f0577abd9df4 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:52.559216 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1005/2020] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:47:52 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:47:52.563064 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9c5403-997f-4a97-9289-83f2d8d85332 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:52.575454 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9c5403-997f-4a97-9289-83f2d8d85332 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:52.581291 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832babed-7352-4e8e-b7e4-12d8055bcb2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:52.582071 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832babed-7352-4e8e-b7e4-12d8055bcb2b admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:52.582603 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832babed-7352-4e8e-b7e4-12d8055bcb2b admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:52.582957 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832babed-7352-4e8e-b7e4-12d8055bcb2b admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:52.583722 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832babed-7352-4e8e-b7e4-12d8055bcb2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:52.585423 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832babed-7352-4e8e-b7e4-12d8055bcb2b admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:52.588710 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:52.603370 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1016/2021] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:47:52 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:47:52.605045 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a2a256-fc34-4e03-bec6-f0577abd9df4 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:52.616180 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a2a256-fc34-4e03-bec6-f0577abd9df4 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:52.622646 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a4ab4d-03d9-4c90-86d1-ac70ee004d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:52.623354 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4ab4d-03d9-4c90-86d1-ac70ee004d07 admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:52.623663 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4ab4d-03d9-4c90-86d1-ac70ee004d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:52.623979 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4ab4d-03d9-4c90-86d1-ac70ee004d07 admin admin] PATH_INFO: `/v3/projects/f82454422ab547af884f7d689ddc9d9f/users/08898a749c3a4be38a452c3fb683a228/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:52.640833 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a4ab4d-03d9-4c90-86d1-ac70ee004d07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f82454422ab547af884f7d689ddc9d9f, user_id=08898a749c3a4be38a452c3fb683a228, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:52.644074 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a4ab4d-03d9-4c90-86d1-ac70ee004d07 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:52.660297 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1006/2022] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:47:52 2022] PUT /identity/v3/projects/f82454422ab547af884f7d689ddc9d9f/users/08898a749c3a4be38a452c3fb683a228/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:47:52.672191 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64848b41-a0e2-4027-b839-b2fdf9a4f689 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:52.672492 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64848b41-a0e2-4027-b839-b2fdf9a4f689 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:52.672804 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64848b41-a0e2-4027-b839-b2fdf9a4f689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:52.702415 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a4ab4d-03d9-4c90-86d1-ac70ee004d07 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:52.703707 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-64848b41-a0e2-4027-b839-b2fdf9a4f689 None None] MFA Rules not processed for user `08898a749c3a4be38a452c3fb683a228`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:47:52.704449 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-01a4ab4d-03d9-4c90-86d1-ac70ee004d07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:52.705103 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-64848b41-a0e2-4027-b839-b2fdf9a4f689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:52.780054 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a4ab4d-03d9-4c90-86d1-ac70ee004d07 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:52.784555 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1017/2023] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:47:52 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:47:52.786190 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a643b4e-b5a4-4f3f-a7bf-a4d1462481d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:52.786923 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a643b4e-b5a4-4f3f-a7bf-a4d1462481d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:52.787508 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a643b4e-b5a4-4f3f-a7bf-a4d1462481d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:52.787673 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a643b4e-b5a4-4f3f-a7bf-a4d1462481d9 admin admin] PATH_INFO: `/v3/projects/ee9e43e71aa9499dbc6a0b2b0807ffa3` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:52.788832 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a643b4e-b5a4-4f3f-a7bf-a4d1462481d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee9e43e71aa9499dbc6a0b2b0807ffa3)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:52.790763 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a643b4e-b5a4-4f3f-a7bf-a4d1462481d9 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:52.796693 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64848b41-a0e2-4027-b839-b2fdf9a4f689 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:52.798315 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-64848b41-a0e2-4027-b839-b2fdf9a4f689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:52.840627 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-9a643b4e-b5a4-4f3f-a7bf-a4d1462481d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee9e43e71aa9499dbc6a0b2b0807ffa3', 'request_id': 'req-9a643b4e-b5a4-4f3f-a7bf-a4d1462481d9'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:52.847887 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-9a643b4e-b5a4-4f3f-a7bf-a4d1462481d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee9e43e71aa9499dbc6a0b2b0807ffa3', 'request_id': 'req-9a643b4e-b5a4-4f3f-a7bf-a4d1462481d9'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:52.857629 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-9a643b4e-b5a4-4f3f-a7bf-a4d1462481d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee9e43e71aa9499dbc6a0b2b0807ffa3', 'request_id': 'req-9a643b4e-b5a4-4f3f-a7bf-a4d1462481d9'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:52.858777 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64848b41-a0e2-4027-b839-b2fdf9a4f689 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:52.870889 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-9a643b4e-b5a4-4f3f-a7bf-a4d1462481d9 admin admin] The token cache is being invalidate because project ee9e43e71aa9499dbc6a0b2b0807ffa3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90653) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:47:52.871273 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-9a643b4e-b5a4-4f3f-a7bf-a4d1462481d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:52.872070 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-9a643b4e-b5a4-4f3f-a7bf-a4d1462481d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:52.877022 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1007/2024] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:47:52 2022] DELETE /identity/v3/projects/ee9e43e71aa9499dbc6a0b2b0807ffa3 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:47:52.878168 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a643b4e-b5a4-4f3f-a7bf-a4d1462481d9 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:52.880173 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-9a643b4e-b5a4-4f3f-a7bf-a4d1462481d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:52.917248 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4516053d-dde8-42dc-84da-c9b8c93695ac service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1632da5fbe384b5c849be1d4d86c0f1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:52.918012 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4516053d-dde8-42dc-84da-c9b8c93695ac service glance] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:52.918275 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4516053d-dde8-42dc-84da-c9b8c93695ac service glance] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:52.918549 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4516053d-dde8-42dc-84da-c9b8c93695ac service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:52.921210 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-4516053d-dde8-42dc-84da-c9b8c93695ac service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:52.962408 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a643b4e-b5a4-4f3f-a7bf-a4d1462481d9 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:52.968490 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff1c7d6-d4fb-4836-90fd-b2ae69b5b583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:52.969383 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff1c7d6-d4fb-4836-90fd-b2ae69b5b583 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:52.969924 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff1c7d6-d4fb-4836-90fd-b2ae69b5b583 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:52.970819 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff1c7d6-d4fb-4836-90fd-b2ae69b5b583 admin admin] PATH_INFO: `/v3/users/8ca90624010448a294c3651f752e209a` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:52.973112 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff1c7d6-d4fb-4836-90fd-b2ae69b5b583 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ca90624010448a294c3651f752e209a)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:52.975490 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff1c7d6-d4fb-4836-90fd-b2ae69b5b583 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:52.977994 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:53.001756 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4516053d-dde8-42dc-84da-c9b8c93695ac service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:53.003968 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4516053d-dde8-42dc-84da-c9b8c93695ac service glance] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:53.009853 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:53.015112 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1018/2025] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:47:52 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 10:47:53.038251 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-8ff1c7d6-d4fb-4836-90fd-b2ae69b5b583 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ca90624010448a294c3651f752e209a'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:53.050885 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-8ff1c7d6-d4fb-4836-90fd-b2ae69b5b583 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ca90624010448a294c3651f752e209a'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:53.053184 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-8ff1c7d6-d4fb-4836-90fd-b2ae69b5b583 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ca90624010448a294c3651f752e209a'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:53.053628 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-8ff1c7d6-d4fb-4836-90fd-b2ae69b5b583 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ca90624010448a294c3651f752e209a'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:53.061610 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-8ff1c7d6-d4fb-4836-90fd-b2ae69b5b583 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ca90624010448a294c3651f752e209a'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:53.082791 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1008/2026] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:47:52 2022] DELETE /identity/v3/users/8ca90624010448a294c3651f752e209a => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 10:47:53.444032 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4516053d-dde8-42dc-84da-c9b8c93695ac service glance] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:53.460137 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4516053d-dde8-42dc-84da-c9b8c93695ac service glance] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:53.559968 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fbaac6-bc7b-4361-96db-e23fdff86799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:53.560719 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fbaac6-bc7b-4361-96db-e23fdff86799 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:53.561133 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fbaac6-bc7b-4361-96db-e23fdff86799 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:53.561684 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fbaac6-bc7b-4361-96db-e23fdff86799 admin admin] PATH_INFO: `/v3/projects/79789607cba2404eae8e49fb4bc3a5e8` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:53.563468 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fbaac6-bc7b-4361-96db-e23fdff86799 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79789607cba2404eae8e49fb4bc3a5e8)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:53.565415 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fbaac6-bc7b-4361-96db-e23fdff86799 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:53.646701 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-35fbaac6-bc7b-4361-96db-e23fdff86799 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79789607cba2404eae8e49fb4bc3a5e8', 'request_id': 'req-35fbaac6-bc7b-4361-96db-e23fdff86799'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:53.656125 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-35fbaac6-bc7b-4361-96db-e23fdff86799 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79789607cba2404eae8e49fb4bc3a5e8', 'request_id': 'req-35fbaac6-bc7b-4361-96db-e23fdff86799'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:53.668637 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-35fbaac6-bc7b-4361-96db-e23fdff86799 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79789607cba2404eae8e49fb4bc3a5e8', 'request_id': 'req-35fbaac6-bc7b-4361-96db-e23fdff86799'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:53.683668 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-35fbaac6-bc7b-4361-96db-e23fdff86799 admin admin] The token cache is being invalidate because project 79789607cba2404eae8e49fb4bc3a5e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:47:53.684086 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-35fbaac6-bc7b-4361-96db-e23fdff86799 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:53.684572 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-35fbaac6-bc7b-4361-96db-e23fdff86799 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:47:53.686334 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1019/2027] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:47:53 2022] DELETE /identity/v3/projects/79789607cba2404eae8e49fb4bc3a5e8 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:47:53.692578 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e546ad-037b-48b1-aca9-44e47168795e None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:53.692875 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e546ad-037b-48b1-aca9-44e47168795e None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:53.693154 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e546ad-037b-48b1-aca9-44e47168795e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:53.722955 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-08e546ad-037b-48b1-aca9-44e47168795e None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:47:53.724704 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-08e546ad-037b-48b1-aca9-44e47168795e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:53.786061 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1009/2028] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:47:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:47:53.794775 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0372ea3-e099-4ade-9e28-c76cb72158bf None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:53.795265 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0372ea3-e099-4ade-9e28-c76cb72158bf None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:53.795366 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0372ea3-e099-4ade-9e28-c76cb72158bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:53.816181 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-f0372ea3-e099-4ade-9e28-c76cb72158bf None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:47:53.817302 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-f0372ea3-e099-4ade-9e28-c76cb72158bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:53.825036 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1020/2029] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:47:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:47:53.830268 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e546ad-037b-48b1-aca9-44e47168795e None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:53.840271 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e546ad-037b-48b1-aca9-44e47168795e None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:53.844929 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa68cad2-7e6c-4d17-b074-95ce8a612869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:53.845526 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa68cad2-7e6c-4d17-b074-95ce8a612869 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:53.845804 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa68cad2-7e6c-4d17-b074-95ce8a612869 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:53.846071 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa68cad2-7e6c-4d17-b074-95ce8a612869 admin admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:53.846712 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa68cad2-7e6c-4d17-b074-95ce8a612869 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:47:53.847030 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa68cad2-7e6c-4d17-b074-95ce8a612869 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:53.848667 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa68cad2-7e6c-4d17-b074-95ce8a612869 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:53.860828 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1010/2030] 10.222.0.15 () {58 vars in 1153 bytes} [Sun Sep 11 10:47:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:47:53.864914 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0372ea3-e099-4ade-9e28-c76cb72158bf None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:53.874367 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0372ea3-e099-4ade-9e28-c76cb72158bf None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:53.879706 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6b7394-0821-46cf-855e-9ffc67a571cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:53.880301 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6b7394-0821-46cf-855e-9ffc67a571cd admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:53.880581 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6b7394-0821-46cf-855e-9ffc67a571cd admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:53.880856 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6b7394-0821-46cf-855e-9ffc67a571cd admin admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:53.881481 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6b7394-0821-46cf-855e-9ffc67a571cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:53.883372 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6b7394-0821-46cf-855e-9ffc67a571cd admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:53.910533 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1021/2031] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:47:53 2022] POST /identity/v3/projects => generated 369 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:47:53.915063 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa68cad2-7e6c-4d17-b074-95ce8a612869 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:53.925883 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa68cad2-7e6c-4d17-b074-95ce8a612869 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:53.981210 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b63dbb1-509c-4034-aad1-f09f277c7597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:53.982080 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b63dbb1-509c-4034-aad1-f09f277c7597 admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:53.982391 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b63dbb1-509c-4034-aad1-f09f277c7597 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:53.982665 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b63dbb1-509c-4034-aad1-f09f277c7597 admin admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:53.983291 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b63dbb1-509c-4034-aad1-f09f277c7597 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:53.985312 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b63dbb1-509c-4034-aad1-f09f277c7597 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:53.996740 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:54.008764 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1011/2032] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:47:53 2022] POST /identity/v3/users => generated 285 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:47:54.013093 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6b7394-0821-46cf-855e-9ffc67a571cd admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:54.023643 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6b7394-0821-46cf-855e-9ffc67a571cd admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:54.028301 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2da13ce-e322-40bf-b3e6-660d18c5185a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:54.028897 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da13ce-e322-40bf-b3e6-660d18c5185a admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:54.029175 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da13ce-e322-40bf-b3e6-660d18c5185a admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:54.029446 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da13ce-e322-40bf-b3e6-660d18c5185a admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:54.030089 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2da13ce-e322-40bf-b3e6-660d18c5185a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:54.031723 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2da13ce-e322-40bf-b3e6-660d18c5185a admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:54.042288 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1022/2033] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:47:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:47:54.047326 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b63dbb1-509c-4034-aad1-f09f277c7597 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:54.057922 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b63dbb1-509c-4034-aad1-f09f277c7597 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:54.062532 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63477a20-3dab-4120-8855-cc4bef8fb5d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:54.063130 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63477a20-3dab-4120-8855-cc4bef8fb5d8 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:54.063412 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63477a20-3dab-4120-8855-cc4bef8fb5d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:54.063717 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63477a20-3dab-4120-8855-cc4bef8fb5d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:54.064347 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63477a20-3dab-4120-8855-cc4bef8fb5d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:54.065924 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63477a20-3dab-4120-8855-cc4bef8fb5d8 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:54.076339 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1012/2034] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:47:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:47:54.080212 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2da13ce-e322-40bf-b3e6-660d18c5185a admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:54.090558 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2da13ce-e322-40bf-b3e6-660d18c5185a admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:54.096565 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e20dcc-8f2a-4fac-a6d8-30459ea54ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:54.097228 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e20dcc-8f2a-4fac-a6d8-30459ea54ae8 admin admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:54.097510 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e20dcc-8f2a-4fac-a6d8-30459ea54ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:54.097786 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e20dcc-8f2a-4fac-a6d8-30459ea54ae8 admin admin] PATH_INFO: `/v3/projects/45a50e1025d949cb8c59366b5646d7b6/users/103e0c1384234f4eac3480026c53cbd8/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:54.113268 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e20dcc-8f2a-4fac-a6d8-30459ea54ae8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45a50e1025d949cb8c59366b5646d7b6, user_id=103e0c1384234f4eac3480026c53cbd8, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:54.115953 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e20dcc-8f2a-4fac-a6d8-30459ea54ae8 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:54.129702 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1023/2035] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:47:54 2022] PUT /identity/v3/projects/45a50e1025d949cb8c59366b5646d7b6/users/103e0c1384234f4eac3480026c53cbd8/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 10:47:54.141977 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106089a2-f157-49a3-be3f-75cf905e58bd None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:54.142528 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106089a2-f157-49a3-be3f-75cf905e58bd None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:54.142881 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106089a2-f157-49a3-be3f-75cf905e58bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:54.172248 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-106089a2-f157-49a3-be3f-75cf905e58bd None None] MFA Rules not processed for user `103e0c1384234f4eac3480026c53cbd8`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:47:54.174174 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-106089a2-f157-49a3-be3f-75cf905e58bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:54.249033 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1013/2036] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:47:54 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:47:54.262845 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e20dcc-8f2a-4fac-a6d8-30459ea54ae8 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:47:54.264640 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-65e20dcc-8f2a-4fac-a6d8-30459ea54ae8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:47:54.331168 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:47:54.362555 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e20dcc-8f2a-4fac-a6d8-30459ea54ae8 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:47:54.366205 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbba5a92-7551-4dff-a2c4-8d4882c1a05e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1632da5fbe384b5c849be1d4d86c0f1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:47:54.366820 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbba5a92-7551-4dff-a2c4-8d4882c1a05e service glance] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:47:54.367101 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbba5a92-7551-4dff-a2c4-8d4882c1a05e service glance] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:47:54.367375 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbba5a92-7551-4dff-a2c4-8d4882c1a05e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:47:54.375641 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbba5a92-7551-4dff-a2c4-8d4882c1a05e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:47:54.377082 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbba5a92-7551-4dff-a2c4-8d4882c1a05e service glance] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:47:54.385843 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1024/2037] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:47:54 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:48:01.742469 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106089a2-f157-49a3-be3f-75cf905e58bd None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:01.744821 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-106089a2-f157-49a3-be3f-75cf905e58bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:01.817918 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106089a2-f157-49a3-be3f-75cf905e58bd None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:01.823120 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44890323-c390-4ccb-9311-00aea3724dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:01.823964 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44890323-c390-4ccb-9311-00aea3724dda admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:01.824319 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44890323-c390-4ccb-9311-00aea3724dda admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:01.825426 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44890323-c390-4ccb-9311-00aea3724dda admin admin] PATH_INFO: `/v3/users/7af4c12ac0bd469aa6c2f1abba485f01` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:01.828296 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44890323-c390-4ccb-9311-00aea3724dda admin admin] RBAC: Authorizing `identity:delete_user(user_id=7af4c12ac0bd469aa6c2f1abba485f01)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:01.830708 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44890323-c390-4ccb-9311-00aea3724dda admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:01.884892 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-44890323-c390-4ccb-9311-00aea3724dda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7af4c12ac0bd469aa6c2f1abba485f01'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:01.893542 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-44890323-c390-4ccb-9311-00aea3724dda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7af4c12ac0bd469aa6c2f1abba485f01'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:01.894172 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-44890323-c390-4ccb-9311-00aea3724dda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7af4c12ac0bd469aa6c2f1abba485f01'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:01.895080 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-44890323-c390-4ccb-9311-00aea3724dda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7af4c12ac0bd469aa6c2f1abba485f01'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:01.908428 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-44890323-c390-4ccb-9311-00aea3724dda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7af4c12ac0bd469aa6c2f1abba485f01'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:01.921304 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:48:01.932143 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1014/2038] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:48:01 2022] DELETE /identity/v3/users/7af4c12ac0bd469aa6c2f1abba485f01 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:48:01.936581 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95b5583-1e41-4984-88bb-ecd718a2e3cd None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:01.937428 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95b5583-1e41-4984-88bb-ecd718a2e3cd None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:01.937707 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95b5583-1e41-4984-88bb-ecd718a2e3cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:01.958322 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-a95b5583-1e41-4984-88bb-ecd718a2e3cd None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:48:01.959669 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-a95b5583-1e41-4984-88bb-ecd718a2e3cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:02.060027 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1025/2039] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:48:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 10:48:02.078572 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44890323-c390-4ccb-9311-00aea3724dda admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:02.080117 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-44890323-c390-4ccb-9311-00aea3724dda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:02.159742 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44890323-c390-4ccb-9311-00aea3724dda admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:02.166079 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aea5e92-fffa-42e8-b4d7-2f6c89dd493a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:02.166929 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aea5e92-fffa-42e8-b4d7-2f6c89dd493a service neutron] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:02.167279 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aea5e92-fffa-42e8-b4d7-2f6c89dd493a service neutron] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:02.167657 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aea5e92-fffa-42e8-b4d7-2f6c89dd493a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:02.179524 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aea5e92-fffa-42e8-b4d7-2f6c89dd493a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:02.181130 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aea5e92-fffa-42e8-b4d7-2f6c89dd493a service neutron] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:02.193145 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1015/2040] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:48:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:48:02.538189 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95b5583-1e41-4984-88bb-ecd718a2e3cd None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:02.542156 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-a95b5583-1e41-4984-88bb-ecd718a2e3cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:02.557789 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95b5583-1e41-4984-88bb-ecd718a2e3cd None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:02.571821 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3858a268-42b6-48df-a9a8-aead2e60835d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:02.571821 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3858a268-42b6-48df-a9a8-aead2e60835d admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:02.571821 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3858a268-42b6-48df-a9a8-aead2e60835d admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:02.571821 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3858a268-42b6-48df-a9a8-aead2e60835d admin admin] PATH_INFO: `/v3/projects/2882defcb26b4707934f2cff1f783f23` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:02.572152 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3858a268-42b6-48df-a9a8-aead2e60835d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2882defcb26b4707934f2cff1f783f23)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:02.572152 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3858a268-42b6-48df-a9a8-aead2e60835d admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:02.638193 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-3858a268-42b6-48df-a9a8-aead2e60835d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2882defcb26b4707934f2cff1f783f23', 'request_id': 'req-3858a268-42b6-48df-a9a8-aead2e60835d'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:02.645578 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-3858a268-42b6-48df-a9a8-aead2e60835d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2882defcb26b4707934f2cff1f783f23', 'request_id': 'req-3858a268-42b6-48df-a9a8-aead2e60835d'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:02.652970 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-3858a268-42b6-48df-a9a8-aead2e60835d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2882defcb26b4707934f2cff1f783f23', 'request_id': 'req-3858a268-42b6-48df-a9a8-aead2e60835d'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:02.669796 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-3858a268-42b6-48df-a9a8-aead2e60835d admin admin] The token cache is being invalidate because project 2882defcb26b4707934f2cff1f783f23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:48:02.670454 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-3858a268-42b6-48df-a9a8-aead2e60835d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:02.671364 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-3858a268-42b6-48df-a9a8-aead2e60835d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:02.674295 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1026/2041] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:48:02 2022] DELETE /identity/v3/projects/2882defcb26b4707934f2cff1f783f23 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:48:02.678416 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aea5e92-fffa-42e8-b4d7-2f6c89dd493a service neutron] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:02.690929 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aea5e92-fffa-42e8-b4d7-2f6c89dd493a service neutron] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:02.763546 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff90da26-8960-44d9-bf00-07c827dc784a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:02.764082 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff90da26-8960-44d9-bf00-07c827dc784a admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:02.764346 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff90da26-8960-44d9-bf00-07c827dc784a admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:02.764564 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff90da26-8960-44d9-bf00-07c827dc784a admin admin] PATH_INFO: `/v3/users/03b64db6b09442fa888388d13cdcdd2d` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:02.765728 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff90da26-8960-44d9-bf00-07c827dc784a admin admin] RBAC: Authorizing `identity:delete_user(user_id=03b64db6b09442fa888388d13cdcdd2d)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:02.768259 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff90da26-8960-44d9-bf00-07c827dc784a admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:02.824411 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-ff90da26-8960-44d9-bf00-07c827dc784a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03b64db6b09442fa888388d13cdcdd2d'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:02.833489 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-ff90da26-8960-44d9-bf00-07c827dc784a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03b64db6b09442fa888388d13cdcdd2d'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:02.841960 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-ff90da26-8960-44d9-bf00-07c827dc784a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03b64db6b09442fa888388d13cdcdd2d'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:02.845360 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-ff90da26-8960-44d9-bf00-07c827dc784a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03b64db6b09442fa888388d13cdcdd2d'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:02.861799 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-ff90da26-8960-44d9-bf00-07c827dc784a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03b64db6b09442fa888388d13cdcdd2d'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:02.881532 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1016/2042] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:48:02 2022] DELETE /identity/v3/users/03b64db6b09442fa888388d13cdcdd2d => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 10:48:04.182806 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3858a268-42b6-48df-a9a8-aead2e60835d admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:04.186285 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-3858a268-42b6-48df-a9a8-aead2e60835d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:04.200444 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3858a268-42b6-48df-a9a8-aead2e60835d admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:04.256418 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:48:04.285301 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a69b583-b281-4c00-80f4-3c79bac9c787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:04.285960 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a69b583-b281-4c00-80f4-3c79bac9c787 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:04.286244 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a69b583-b281-4c00-80f4-3c79bac9c787 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:04.286520 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a69b583-b281-4c00-80f4-3c79bac9c787 admin admin] PATH_INFO: `/v3/projects/2c1b35454b884624a9246994e2545d01` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:04.287741 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a69b583-b281-4c00-80f4-3c79bac9c787 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c1b35454b884624a9246994e2545d01)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:04.289692 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a69b583-b281-4c00-80f4-3c79bac9c787 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:04.356028 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-8a69b583-b281-4c00-80f4-3c79bac9c787 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c1b35454b884624a9246994e2545d01', 'request_id': 'req-8a69b583-b281-4c00-80f4-3c79bac9c787'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:04.364572 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-8a69b583-b281-4c00-80f4-3c79bac9c787 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c1b35454b884624a9246994e2545d01', 'request_id': 'req-8a69b583-b281-4c00-80f4-3c79bac9c787'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:04.371600 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-8a69b583-b281-4c00-80f4-3c79bac9c787 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c1b35454b884624a9246994e2545d01', 'request_id': 'req-8a69b583-b281-4c00-80f4-3c79bac9c787'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:04.390617 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-8a69b583-b281-4c00-80f4-3c79bac9c787 admin admin] The token cache is being invalidate because project 2c1b35454b884624a9246994e2545d01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:48:04.390829 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-8a69b583-b281-4c00-80f4-3c79bac9c787 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:04.391591 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-8a69b583-b281-4c00-80f4-3c79bac9c787 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:04.404894 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1027/2043] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:48:04 2022] DELETE /identity/v3/projects/2c1b35454b884624a9246994e2545d01 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:48:04.405015 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34445a-4356-4709-8a4c-eb3e4a6ece0f None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:04.405015 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34445a-4356-4709-8a4c-eb3e4a6ece0f None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:04.405015 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34445a-4356-4709-8a4c-eb3e4a6ece0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:04.423702 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-7f34445a-4356-4709-8a4c-eb3e4a6ece0f None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:48:04.428092 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-7f34445a-4356-4709-8a4c-eb3e4a6ece0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:04.440658 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:48:04.497767 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1017/2044] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:48:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:48:04.506238 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a18c662-b590-473a-bb94-6537f7e4d065 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:04.506650 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a18c662-b590-473a-bb94-6537f7e4d065 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:04.506792 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a18c662-b590-473a-bb94-6537f7e4d065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:04.538314 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-8a18c662-b590-473a-bb94-6537f7e4d065 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:48:04.539837 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-8a18c662-b590-473a-bb94-6537f7e4d065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:04.549029 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1028/2045] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:48:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:48:04.553289 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f34445a-4356-4709-8a4c-eb3e4a6ece0f None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:04.563416 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f34445a-4356-4709-8a4c-eb3e4a6ece0f None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:04.568928 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ffb5d3-51c8-44e2-98a5-a24d48bba1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:04.569480 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ffb5d3-51c8-44e2-98a5-a24d48bba1bf admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:04.569767 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ffb5d3-51c8-44e2-98a5-a24d48bba1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:04.570042 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ffb5d3-51c8-44e2-98a5-a24d48bba1bf admin admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:04.570703 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ffb5d3-51c8-44e2-98a5-a24d48bba1bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:48:04.571010 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ffb5d3-51c8-44e2-98a5-a24d48bba1bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:04.572692 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ffb5d3-51c8-44e2-98a5-a24d48bba1bf admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:04.587460 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1018/2046] 10.222.0.15 () {58 vars in 1153 bytes} [Sun Sep 11 10:48:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:48:04.590085 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a18c662-b590-473a-bb94-6537f7e4d065 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:04.600053 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a18c662-b590-473a-bb94-6537f7e4d065 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:04.604954 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c175b06-b3e0-42d9-8eb7-d4c90c0a3723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:04.605566 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c175b06-b3e0-42d9-8eb7-d4c90c0a3723 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:04.605847 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c175b06-b3e0-42d9-8eb7-d4c90c0a3723 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:04.606146 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c175b06-b3e0-42d9-8eb7-d4c90c0a3723 admin admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:04.607035 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c175b06-b3e0-42d9-8eb7-d4c90c0a3723 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:04.608999 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c175b06-b3e0-42d9-8eb7-d4c90c0a3723 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:04.643175 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1029/2047] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:48:04 2022] POST /identity/v3/projects => generated 409 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:48:04.644500 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ffb5d3-51c8-44e2-98a5-a24d48bba1bf admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:04.654719 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ffb5d3-51c8-44e2-98a5-a24d48bba1bf admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:04.724602 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf968a21-aa8b-4352-8c74-6fa0f41c1cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:04.725364 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf968a21-aa8b-4352-8c74-6fa0f41c1cf4 admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:04.725781 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf968a21-aa8b-4352-8c74-6fa0f41c1cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:04.726080 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf968a21-aa8b-4352-8c74-6fa0f41c1cf4 admin admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:04.726737 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf968a21-aa8b-4352-8c74-6fa0f41c1cf4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:04.728828 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf968a21-aa8b-4352-8c74-6fa0f41c1cf4 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:04.754484 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1019/2048] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:48:04 2022] POST /identity/v3/users => generated 305 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:48:04.755963 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c175b06-b3e0-42d9-8eb7-d4c90c0a3723 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:04.767821 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c175b06-b3e0-42d9-8eb7-d4c90c0a3723 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:04.775254 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cfd1d8-08e1-4830-861a-4ba0d3dfb64e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:04.775986 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cfd1d8-08e1-4830-861a-4ba0d3dfb64e admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:04.776346 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cfd1d8-08e1-4830-861a-4ba0d3dfb64e admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:04.776622 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cfd1d8-08e1-4830-861a-4ba0d3dfb64e admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:04.777295 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cfd1d8-08e1-4830-861a-4ba0d3dfb64e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:04.778994 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cfd1d8-08e1-4830-861a-4ba0d3dfb64e admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:04.792087 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1030/2049] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:48:04 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:48:04.796276 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf968a21-aa8b-4352-8c74-6fa0f41c1cf4 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:04.808013 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf968a21-aa8b-4352-8c74-6fa0f41c1cf4 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:04.813590 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee5506f-c74b-473f-b12e-5459cbd34653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:04.814593 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5506f-c74b-473f-b12e-5459cbd34653 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:04.815067 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5506f-c74b-473f-b12e-5459cbd34653 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:04.815413 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5506f-c74b-473f-b12e-5459cbd34653 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:04.816187 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee5506f-c74b-473f-b12e-5459cbd34653 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:04.817963 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee5506f-c74b-473f-b12e-5459cbd34653 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:04.831130 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1020/2050] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:48:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 10:48:04.835200 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cfd1d8-08e1-4830-861a-4ba0d3dfb64e admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:04.851488 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cfd1d8-08e1-4830-861a-4ba0d3dfb64e admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:04.857460 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a4d7ff-63e2-4e30-a984-7cf55c10dc33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:04.858135 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a4d7ff-63e2-4e30-a984-7cf55c10dc33 admin admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:04.858418 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a4d7ff-63e2-4e30-a984-7cf55c10dc33 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:04.858697 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a4d7ff-63e2-4e30-a984-7cf55c10dc33 admin admin] PATH_INFO: `/v3/projects/60c6ab6ecd584283b90d4cf4b122fe17/users/6e3bc3a71e954ce7a82e569d08aa529e/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:04.887632 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a4d7ff-63e2-4e30-a984-7cf55c10dc33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60c6ab6ecd584283b90d4cf4b122fe17, user_id=6e3bc3a71e954ce7a82e569d08aa529e, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:04.894469 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a4d7ff-63e2-4e30-a984-7cf55c10dc33 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:04.919501 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1031/2051] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:48:04 2022] PUT /identity/v3/projects/60c6ab6ecd584283b90d4cf4b122fe17/users/6e3bc3a71e954ce7a82e569d08aa529e/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 10:48:04.942943 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d95ea5-50e6-4c25-9b66-98445524ed3e None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:04.944210 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d95ea5-50e6-4c25-9b66-98445524ed3e None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:04.944763 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d95ea5-50e6-4c25-9b66-98445524ed3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:04.976233 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-a2d95ea5-50e6-4c25-9b66-98445524ed3e None None] MFA Rules not processed for user `6e3bc3a71e954ce7a82e569d08aa529e`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:48:04.977469 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-a2d95ea5-50e6-4c25-9b66-98445524ed3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:05.057685 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1021/2052] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:48:04 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:48:05.061559 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a4d7ff-63e2-4e30-a984-7cf55c10dc33 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:05.147199 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a4d7ff-63e2-4e30-a984-7cf55c10dc33 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:05.152555 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb76e36-cac0-420d-9e57-793c0fa1e95e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:05.153209 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb76e36-cac0-420d-9e57-793c0fa1e95e admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:05.153552 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb76e36-cac0-420d-9e57-793c0fa1e95e admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:05.153845 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb76e36-cac0-420d-9e57-793c0fa1e95e admin admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:05.154506 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb76e36-cac0-420d-9e57-793c0fa1e95e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:05.156663 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb76e36-cac0-420d-9e57-793c0fa1e95e admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:05.189482 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1032/2053] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:48:05 2022] POST /identity/v3/projects => generated 409 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:48:05.193032 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d95ea5-50e6-4c25-9b66-98445524ed3e None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:05.203861 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d95ea5-50e6-4c25-9b66-98445524ed3e None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:05.211960 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:48:05.265610 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b365ec2-489c-417f-ae65-050349e9dc89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:05.266261 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b365ec2-489c-417f-ae65-050349e9dc89 admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:05.266545 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b365ec2-489c-417f-ae65-050349e9dc89 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:05.266819 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b365ec2-489c-417f-ae65-050349e9dc89 admin admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:05.267471 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b365ec2-489c-417f-ae65-050349e9dc89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:05.269402 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b365ec2-489c-417f-ae65-050349e9dc89 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:05.302821 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1022/2054] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:48:05 2022] POST /identity/v3/users => generated 311 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 10:48:05.304175 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb76e36-cac0-420d-9e57-793c0fa1e95e admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:05.311658 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:48:05.319359 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb76e36-cac0-420d-9e57-793c0fa1e95e admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:05.327208 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7029af-4b9f-46e8-9722-f167034e7309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:05.328301 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7029af-4b9f-46e8-9722-f167034e7309 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:05.329593 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7029af-4b9f-46e8-9722-f167034e7309 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:05.330230 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7029af-4b9f-46e8-9722-f167034e7309 admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:05.331161 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7029af-4b9f-46e8-9722-f167034e7309 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:05.335481 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7029af-4b9f-46e8-9722-f167034e7309 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:05.359138 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1033/2055] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:48:05 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:48:05.360339 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b365ec2-489c-417f-ae65-050349e9dc89 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:05.370914 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b365ec2-489c-417f-ae65-050349e9dc89 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:05.377634 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa3364e-66fd-4950-97e5-143e9f6c6ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:05.378318 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa3364e-66fd-4950-97e5-143e9f6c6ef0 admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:05.378603 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa3364e-66fd-4950-97e5-143e9f6c6ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:05.378881 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa3364e-66fd-4950-97e5-143e9f6c6ef0 admin admin] PATH_INFO: `/v3/projects/1ddf68d92d91429b9708b1db22f027a6/users/94753dc0bdf34d60a2632b1901c3cc55/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:05.397526 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa3364e-66fd-4950-97e5-143e9f6c6ef0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ddf68d92d91429b9708b1db22f027a6, user_id=94753dc0bdf34d60a2632b1901c3cc55, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:05.400551 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa3364e-66fd-4950-97e5-143e9f6c6ef0 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:05.421308 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1023/2056] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:48:05 2022] PUT /identity/v3/projects/1ddf68d92d91429b9708b1db22f027a6/users/94753dc0bdf34d60a2632b1901c3cc55/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:48:05.425143 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7029af-4b9f-46e8-9722-f167034e7309 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:05.440958 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7029af-4b9f-46e8-9722-f167034e7309 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:05.592287 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22137f34-efed-4801-aa8d-2cf3e2348417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:05.593032 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22137f34-efed-4801-aa8d-2cf3e2348417 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:05.593382 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22137f34-efed-4801-aa8d-2cf3e2348417 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:05.593743 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22137f34-efed-4801-aa8d-2cf3e2348417 admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:05.594721 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22137f34-efed-4801-aa8d-2cf3e2348417 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:05.596635 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22137f34-efed-4801-aa8d-2cf3e2348417 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:05.619635 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1034/2057] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:48:05 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:48:05.620894 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa3364e-66fd-4950-97e5-143e9f6c6ef0 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:05.632614 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa3364e-66fd-4950-97e5-143e9f6c6ef0 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:05.640310 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a07a16a-319f-4106-9f89-4c4dc651cdd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:05.641704 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a07a16a-319f-4106-9f89-4c4dc651cdd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:05.641956 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a07a16a-319f-4106-9f89-4c4dc651cdd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:05.642277 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a07a16a-319f-4106-9f89-4c4dc651cdd4 admin admin] PATH_INFO: `/v3/projects/1ddf68d92d91429b9708b1db22f027a6/users/94753dc0bdf34d60a2632b1901c3cc55/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:05.644575 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a07a16a-319f-4106-9f89-4c4dc651cdd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ddf68d92d91429b9708b1db22f027a6, user_id=94753dc0bdf34d60a2632b1901c3cc55, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:05.647965 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a07a16a-319f-4106-9f89-4c4dc651cdd4 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:05.670116 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1024/2058] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:48:05 2022] PUT /identity/v3/projects/1ddf68d92d91429b9708b1db22f027a6/users/94753dc0bdf34d60a2632b1901c3cc55/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:48:05.673149 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78468b10-c172-4714-bcdc-6bb5c92ad6d2 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:05.674717 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78468b10-c172-4714-bcdc-6bb5c92ad6d2 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:05.674717 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78468b10-c172-4714-bcdc-6bb5c92ad6d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:05.710278 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-78468b10-c172-4714-bcdc-6bb5c92ad6d2 None None] MFA Rules not processed for user `94753dc0bdf34d60a2632b1901c3cc55`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:48:05.711747 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-78468b10-c172-4714-bcdc-6bb5c92ad6d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:05.795515 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1035/2059] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:48:05 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:48:05.807163 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a07a16a-319f-4106-9f89-4c4dc651cdd4 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:05.809180 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-0a07a16a-319f-4106-9f89-4c4dc651cdd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:05.892245 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a07a16a-319f-4106-9f89-4c4dc651cdd4 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:05.897414 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d20906a-30bf-43f5-b156-aa0a59a647a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:05.898115 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d20906a-30bf-43f5-b156-aa0a59a647a0 service cinder] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:05.898422 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d20906a-30bf-43f5-b156-aa0a59a647a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:05.898699 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d20906a-30bf-43f5-b156-aa0a59a647a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:05.912867 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d20906a-30bf-43f5-b156-aa0a59a647a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:05.914706 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d20906a-30bf-43f5-b156-aa0a59a647a0 service cinder] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:05.930009 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1025/2060] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:48:05 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:48:06.113044 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78468b10-c172-4714-bcdc-6bb5c92ad6d2 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:06.114492 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-78468b10-c172-4714-bcdc-6bb5c92ad6d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:06.129052 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78468b10-c172-4714-bcdc-6bb5c92ad6d2 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:06.132918 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3c6822-58f9-42a6-8a73-48c6f9c58422 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:06.133858 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3c6822-58f9-42a6-8a73-48c6f9c58422 service cinder] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:06.134263 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3c6822-58f9-42a6-8a73-48c6f9c58422 service cinder] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:06.134572 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3c6822-58f9-42a6-8a73-48c6f9c58422 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:06.203500 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3c6822-58f9-42a6-8a73-48c6f9c58422 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:06.205794 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3c6822-58f9-42a6-8a73-48c6f9c58422 service cinder] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:06.216500 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1036/2061] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:48:06 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:48:15.922144 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=65,keystone:INSERT=5 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:48:16.211310 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=95 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:48:18.942244 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d20906a-30bf-43f5-b156-aa0a59a647a0 service cinder] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:19.008835 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d20906a-30bf-43f5-b156-aa0a59a647a0 service cinder] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:19.013578 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffdf8dd-b500-4441-ad92-bd0ced662216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:19.014246 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffdf8dd-b500-4441-ad92-bd0ced662216 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:19.014526 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffdf8dd-b500-4441-ad92-bd0ced662216 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:19.014799 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffdf8dd-b500-4441-ad92-bd0ced662216 admin admin] PATH_INFO: `/v3/users/6e3bc3a71e954ce7a82e569d08aa529e` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:19.016030 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffdf8dd-b500-4441-ad92-bd0ced662216 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e3bc3a71e954ce7a82e569d08aa529e)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:19.017955 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffdf8dd-b500-4441-ad92-bd0ced662216 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:19.066488 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-dffdf8dd-b500-4441-ad92-bd0ced662216 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e3bc3a71e954ce7a82e569d08aa529e'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:19.073659 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-dffdf8dd-b500-4441-ad92-bd0ced662216 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e3bc3a71e954ce7a82e569d08aa529e'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:19.074304 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-dffdf8dd-b500-4441-ad92-bd0ced662216 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e3bc3a71e954ce7a82e569d08aa529e'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:19.074976 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-dffdf8dd-b500-4441-ad92-bd0ced662216 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e3bc3a71e954ce7a82e569d08aa529e'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:19.083281 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-dffdf8dd-b500-4441-ad92-bd0ced662216 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e3bc3a71e954ce7a82e569d08aa529e'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:19.103583 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1026/2062] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:48:18 2022] DELETE /identity/v3/users/6e3bc3a71e954ce7a82e569d08aa529e => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:48:19.108418 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5fff62-ee04-422f-b92c-2e104d79d461 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:19.108882 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5fff62-ee04-422f-b92c-2e104d79d461 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:19.108974 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5fff62-ee04-422f-b92c-2e104d79d461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:19.128306 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-3e5fff62-ee04-422f-b92c-2e104d79d461 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:48:19.129526 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-3e5fff62-ee04-422f-b92c-2e104d79d461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:19.188103 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1037/2063] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:48:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:48:19.197854 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffdf8dd-b500-4441-ad92-bd0ced662216 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:19.199266 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-dffdf8dd-b500-4441-ad92-bd0ced662216 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:19.259605 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffdf8dd-b500-4441-ad92-bd0ced662216 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:19.263540 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07346eef-106f-4a87-816d-afa707621c65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:19.264151 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07346eef-106f-4a87-816d-afa707621c65 service neutron] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:19.264433 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07346eef-106f-4a87-816d-afa707621c65 service neutron] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:19.264708 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07346eef-106f-4a87-816d-afa707621c65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:19.279051 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07346eef-106f-4a87-816d-afa707621c65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:19.280663 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07346eef-106f-4a87-816d-afa707621c65 service neutron] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:19.292518 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1027/2064] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:48:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:48:19.692290 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5fff62-ee04-422f-b92c-2e104d79d461 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:19.693926 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-3e5fff62-ee04-422f-b92c-2e104d79d461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:19.713349 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5fff62-ee04-422f-b92c-2e104d79d461 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:19.718864 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634c037d-924a-42f1-ae16-f6fc056a25a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:19.719728 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634c037d-924a-42f1-ae16-f6fc056a25a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:19.720024 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634c037d-924a-42f1-ae16-f6fc056a25a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:19.720300 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634c037d-924a-42f1-ae16-f6fc056a25a2 admin admin] PATH_INFO: `/v3/projects/60c6ab6ecd584283b90d4cf4b122fe17` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:19.721428 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634c037d-924a-42f1-ae16-f6fc056a25a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60c6ab6ecd584283b90d4cf4b122fe17)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:19.723327 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634c037d-924a-42f1-ae16-f6fc056a25a2 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:19.772819 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-634c037d-924a-42f1-ae16-f6fc056a25a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60c6ab6ecd584283b90d4cf4b122fe17', 'request_id': 'req-634c037d-924a-42f1-ae16-f6fc056a25a2'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:19.779102 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-634c037d-924a-42f1-ae16-f6fc056a25a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60c6ab6ecd584283b90d4cf4b122fe17', 'request_id': 'req-634c037d-924a-42f1-ae16-f6fc056a25a2'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:19.785235 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-634c037d-924a-42f1-ae16-f6fc056a25a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60c6ab6ecd584283b90d4cf4b122fe17', 'request_id': 'req-634c037d-924a-42f1-ae16-f6fc056a25a2'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:19.798561 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-634c037d-924a-42f1-ae16-f6fc056a25a2 admin admin] The token cache is being invalidate because project 60c6ab6ecd584283b90d4cf4b122fe17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:48:19.798859 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-634c037d-924a-42f1-ae16-f6fc056a25a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:19.799573 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-634c037d-924a-42f1-ae16-f6fc056a25a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:19.801114 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1038/2065] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:48:19 2022] DELETE /identity/v3/projects/60c6ab6ecd584283b90d4cf4b122fe17 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:48:19.805193 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07346eef-106f-4a87-816d-afa707621c65 service neutron] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:19.815356 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07346eef-106f-4a87-816d-afa707621c65 service neutron] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:19.835929 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:48:19.898394 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb69f6a-2213-4e3a-811f-0ac13cbb0782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:19.899097 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb69f6a-2213-4e3a-811f-0ac13cbb0782 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:19.899346 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb69f6a-2213-4e3a-811f-0ac13cbb0782 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:19.899672 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb69f6a-2213-4e3a-811f-0ac13cbb0782 admin admin] PATH_INFO: `/v3/users/94753dc0bdf34d60a2632b1901c3cc55` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:19.900877 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb69f6a-2213-4e3a-811f-0ac13cbb0782 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94753dc0bdf34d60a2632b1901c3cc55)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:19.902804 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb69f6a-2213-4e3a-811f-0ac13cbb0782 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:19.951242 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-9cb69f6a-2213-4e3a-811f-0ac13cbb0782 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94753dc0bdf34d60a2632b1901c3cc55'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:19.959253 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-9cb69f6a-2213-4e3a-811f-0ac13cbb0782 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94753dc0bdf34d60a2632b1901c3cc55'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:19.959929 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-9cb69f6a-2213-4e3a-811f-0ac13cbb0782 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94753dc0bdf34d60a2632b1901c3cc55'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:19.960548 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-9cb69f6a-2213-4e3a-811f-0ac13cbb0782 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94753dc0bdf34d60a2632b1901c3cc55'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:19.967336 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-9cb69f6a-2213-4e3a-811f-0ac13cbb0782 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94753dc0bdf34d60a2632b1901c3cc55'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:19.987143 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1028/2066] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:48:19 2022] DELETE /identity/v3/users/94753dc0bdf34d60a2632b1901c3cc55 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:48:20.263069 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634c037d-924a-42f1-ae16-f6fc056a25a2 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:20.264620 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-634c037d-924a-42f1-ae16-f6fc056a25a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:20.281331 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634c037d-924a-42f1-ae16-f6fc056a25a2 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:20.351844 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d962fcf9-01a1-44be-b7ef-3ea1a88e7172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:20.352482 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962fcf9-01a1-44be-b7ef-3ea1a88e7172 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:20.352758 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962fcf9-01a1-44be-b7ef-3ea1a88e7172 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:20.353028 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962fcf9-01a1-44be-b7ef-3ea1a88e7172 admin admin] PATH_INFO: `/v3/projects/1ddf68d92d91429b9708b1db22f027a6` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:20.354114 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d962fcf9-01a1-44be-b7ef-3ea1a88e7172 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ddf68d92d91429b9708b1db22f027a6)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:20.356239 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d962fcf9-01a1-44be-b7ef-3ea1a88e7172 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:20.388787 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:48:20.409222 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-d962fcf9-01a1-44be-b7ef-3ea1a88e7172 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ddf68d92d91429b9708b1db22f027a6', 'request_id': 'req-d962fcf9-01a1-44be-b7ef-3ea1a88e7172'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:20.415764 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-d962fcf9-01a1-44be-b7ef-3ea1a88e7172 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ddf68d92d91429b9708b1db22f027a6', 'request_id': 'req-d962fcf9-01a1-44be-b7ef-3ea1a88e7172'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:20.421205 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-d962fcf9-01a1-44be-b7ef-3ea1a88e7172 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ddf68d92d91429b9708b1db22f027a6', 'request_id': 'req-d962fcf9-01a1-44be-b7ef-3ea1a88e7172'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:20.435335 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-d962fcf9-01a1-44be-b7ef-3ea1a88e7172 admin admin] The token cache is being invalidate because project 1ddf68d92d91429b9708b1db22f027a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:48:20.435654 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-d962fcf9-01a1-44be-b7ef-3ea1a88e7172 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:20.436274 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-d962fcf9-01a1-44be-b7ef-3ea1a88e7172 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:20.438381 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1039/2067] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:48:20 2022] DELETE /identity/v3/projects/1ddf68d92d91429b9708b1db22f027a6 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:48:20.443538 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4909105f-77dc-4992-95fb-924b3602a1b4 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:20.443829 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4909105f-77dc-4992-95fb-924b3602a1b4 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:20.444107 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4909105f-77dc-4992-95fb-924b3602a1b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:20.463750 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-4909105f-77dc-4992-95fb-924b3602a1b4 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:48:20.465011 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-4909105f-77dc-4992-95fb-924b3602a1b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:20.530252 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1029/2068] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:48:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:48:20.539759 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff94aa14-3b10-452d-86e6-da04821245b9 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:20.540103 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff94aa14-3b10-452d-86e6-da04821245b9 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:20.540387 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff94aa14-3b10-452d-86e6-da04821245b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:20.560341 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-ff94aa14-3b10-452d-86e6-da04821245b9 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:48:20.561559 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-ff94aa14-3b10-452d-86e6-da04821245b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:20.569725 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1040/2069] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:48:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:48:20.574619 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4909105f-77dc-4992-95fb-924b3602a1b4 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:20.584776 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4909105f-77dc-4992-95fb-924b3602a1b4 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:20.589470 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ab1779-964b-426c-9262-e8cb530dc1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:20.590116 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ab1779-964b-426c-9262-e8cb530dc1d5 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:20.590397 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ab1779-964b-426c-9262-e8cb530dc1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:20.590667 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ab1779-964b-426c-9262-e8cb530dc1d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:20.591317 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ab1779-964b-426c-9262-e8cb530dc1d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:48:20.591657 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ab1779-964b-426c-9262-e8cb530dc1d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:20.593265 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ab1779-964b-426c-9262-e8cb530dc1d5 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:20.605659 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1030/2070] 10.222.0.15 () {58 vars in 1153 bytes} [Sun Sep 11 10:48:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:48:20.610987 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff94aa14-3b10-452d-86e6-da04821245b9 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:20.620217 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff94aa14-3b10-452d-86e6-da04821245b9 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:20.624836 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7101b56d-8a6a-4603-b2cb-bbf1c0b78cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:20.625930 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7101b56d-8a6a-4603-b2cb-bbf1c0b78cb5 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:20.626214 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7101b56d-8a6a-4603-b2cb-bbf1c0b78cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:20.626490 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7101b56d-8a6a-4603-b2cb-bbf1c0b78cb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:20.627116 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7101b56d-8a6a-4603-b2cb-bbf1c0b78cb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:20.629086 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7101b56d-8a6a-4603-b2cb-bbf1c0b78cb5 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:20.661730 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1041/2071] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:48:20 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 10:48:20.662997 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ab1779-964b-426c-9262-e8cb530dc1d5 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:20.673850 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ab1779-964b-426c-9262-e8cb530dc1d5 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:20.704974 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:48:20.758831 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed1e6b5-3cfe-445f-8f0d-368ab4675eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:20.759527 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed1e6b5-3cfe-445f-8f0d-368ab4675eb3 admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:20.759767 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed1e6b5-3cfe-445f-8f0d-368ab4675eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:20.760059 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed1e6b5-3cfe-445f-8f0d-368ab4675eb3 admin admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:20.760699 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed1e6b5-3cfe-445f-8f0d-368ab4675eb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:20.762638 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed1e6b5-3cfe-445f-8f0d-368ab4675eb3 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:20.788833 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1031/2072] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:48:20 2022] POST /identity/v3/users => generated 293 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:48:20.790109 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7101b56d-8a6a-4603-b2cb-bbf1c0b78cb5 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:20.800792 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7101b56d-8a6a-4603-b2cb-bbf1c0b78cb5 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:20.805980 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91faeb76-9d00-4047-b592-f12e7ef50fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:20.806595 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91faeb76-9d00-4047-b592-f12e7ef50fbe admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:20.807019 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91faeb76-9d00-4047-b592-f12e7ef50fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:20.807304 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91faeb76-9d00-4047-b592-f12e7ef50fbe admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:20.807976 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91faeb76-9d00-4047-b592-f12e7ef50fbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:20.809567 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91faeb76-9d00-4047-b592-f12e7ef50fbe admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:20.825502 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1042/2073] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:48:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 10:48:20.826724 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed1e6b5-3cfe-445f-8f0d-368ab4675eb3 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:20.837755 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed1e6b5-3cfe-445f-8f0d-368ab4675eb3 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:20.842458 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ca5b1f-4636-40f9-a91a-0fed56642e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:20.843054 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ca5b1f-4636-40f9-a91a-0fed56642e0b admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:20.843338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ca5b1f-4636-40f9-a91a-0fed56642e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:20.843648 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ca5b1f-4636-40f9-a91a-0fed56642e0b admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:20.844735 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ca5b1f-4636-40f9-a91a-0fed56642e0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:20.846547 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ca5b1f-4636-40f9-a91a-0fed56642e0b admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:20.857948 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1032/2074] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:48:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:48:20.862400 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91faeb76-9d00-4047-b592-f12e7ef50fbe admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:20.872535 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91faeb76-9d00-4047-b592-f12e7ef50fbe admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:20.877919 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0645ab19-0015-4432-aa64-877051db6d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:20.878498 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0645ab19-0015-4432-aa64-877051db6d64 admin admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:20.878778 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0645ab19-0015-4432-aa64-877051db6d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:20.879082 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0645ab19-0015-4432-aa64-877051db6d64 admin admin] PATH_INFO: `/v3/projects/2000cce93af94b9c8422852c55fc6433/users/fc0271cc3e3c4430aa20e714067cacc6/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:20.894865 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0645ab19-0015-4432-aa64-877051db6d64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2000cce93af94b9c8422852c55fc6433, user_id=fc0271cc3e3c4430aa20e714067cacc6, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:20.899240 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0645ab19-0015-4432-aa64-877051db6d64 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:20.915531 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1043/2075] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:48:20 2022] PUT /identity/v3/projects/2000cce93af94b9c8422852c55fc6433/users/fc0271cc3e3c4430aa20e714067cacc6/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 10:48:20.926447 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb94877-28ce-4631-9557-633f99ed03f3 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:20.926768 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb94877-28ce-4631-9557-633f99ed03f3 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:20.927046 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb94877-28ce-4631-9557-633f99ed03f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:20.957618 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-4eb94877-28ce-4631-9557-633f99ed03f3 None None] MFA Rules not processed for user `fc0271cc3e3c4430aa20e714067cacc6`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:48:20.958839 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-4eb94877-28ce-4631-9557-633f99ed03f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:21.024255 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1033/2076] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:48:20 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:48:21.035072 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0645ab19-0015-4432-aa64-877051db6d64 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:21.036839 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-0645ab19-0015-4432-aa64-877051db6d64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:21.098263 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0645ab19-0015-4432-aa64-877051db6d64 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:21.101966 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95297079-ae23-4989-a610-8c9e38ee09b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:21.102611 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95297079-ae23-4989-a610-8c9e38ee09b6 service cinder] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:21.102896 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95297079-ae23-4989-a610-8c9e38ee09b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:21.103169 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95297079-ae23-4989-a610-8c9e38ee09b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:21.112983 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95297079-ae23-4989-a610-8c9e38ee09b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:21.114417 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95297079-ae23-4989-a610-8c9e38ee09b6 service cinder] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:21.122989 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1044/2077] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:48:21 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:48:31.021155 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=53,keystone:INSERT=3 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:48:31.119100 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:DELETE=6,keystone:SELECT=67,keystone:INSERT=3 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:48:33.754841 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb94877-28ce-4631-9557-633f99ed03f3 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:33.821706 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb94877-28ce-4631-9557-633f99ed03f3 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:33.826638 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812a86e0-459d-49ae-af22-efee59038065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:33.826942 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a86e0-459d-49ae-af22-efee59038065 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:33.827220 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a86e0-459d-49ae-af22-efee59038065 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:33.827537 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a86e0-459d-49ae-af22-efee59038065 admin admin] PATH_INFO: `/v3/users/fc0271cc3e3c4430aa20e714067cacc6` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:33.828725 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812a86e0-459d-49ae-af22-efee59038065 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc0271cc3e3c4430aa20e714067cacc6)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:33.830835 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812a86e0-459d-49ae-af22-efee59038065 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:33.883820 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-812a86e0-459d-49ae-af22-efee59038065 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc0271cc3e3c4430aa20e714067cacc6'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:33.891999 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-812a86e0-459d-49ae-af22-efee59038065 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc0271cc3e3c4430aa20e714067cacc6'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:33.892663 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-812a86e0-459d-49ae-af22-efee59038065 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc0271cc3e3c4430aa20e714067cacc6'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:33.893278 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-812a86e0-459d-49ae-af22-efee59038065 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc0271cc3e3c4430aa20e714067cacc6'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:33.900792 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-812a86e0-459d-49ae-af22-efee59038065 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc0271cc3e3c4430aa20e714067cacc6'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:33.919156 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1034/2078] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:48:33 2022] DELETE /identity/v3/users/fc0271cc3e3c4430aa20e714067cacc6 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:48:33.924312 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4aa738-53d7-4024-b003-948eb5304630 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:33.924815 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4aa738-53d7-4024-b003-948eb5304630 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:33.924991 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4aa738-53d7-4024-b003-948eb5304630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:33.944704 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-0e4aa738-53d7-4024-b003-948eb5304630 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:48:33.945909 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-0e4aa738-53d7-4024-b003-948eb5304630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:34.005351 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1045/2079] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:48:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:48:34.014997 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812a86e0-459d-49ae-af22-efee59038065 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:34.017037 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-812a86e0-459d-49ae-af22-efee59038065 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:34.076816 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812a86e0-459d-49ae-af22-efee59038065 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:34.084558 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff88bc3-5cf1-4dda-b369-b34a31b2391a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:34.085241 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff88bc3-5cf1-4dda-b369-b34a31b2391a service neutron] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:34.085549 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff88bc3-5cf1-4dda-b369-b34a31b2391a service neutron] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:34.085868 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff88bc3-5cf1-4dda-b369-b34a31b2391a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:34.095669 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff88bc3-5cf1-4dda-b369-b34a31b2391a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:34.097257 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff88bc3-5cf1-4dda-b369-b34a31b2391a service neutron] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:34.106841 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1035/2080] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:48:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:48:34.376786 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4aa738-53d7-4024-b003-948eb5304630 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:34.378237 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-0e4aa738-53d7-4024-b003-948eb5304630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:34.390074 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4aa738-53d7-4024-b003-948eb5304630 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:34.395409 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4c7c32-3393-4cbc-8f08-fda98a85364b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:34.396302 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4c7c32-3393-4cbc-8f08-fda98a85364b admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:34.396627 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4c7c32-3393-4cbc-8f08-fda98a85364b admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:34.396981 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4c7c32-3393-4cbc-8f08-fda98a85364b admin admin] PATH_INFO: `/v3/projects/2000cce93af94b9c8422852c55fc6433` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:34.398093 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4c7c32-3393-4cbc-8f08-fda98a85364b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2000cce93af94b9c8422852c55fc6433)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:34.401155 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4c7c32-3393-4cbc-8f08-fda98a85364b admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:34.448177 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-ea4c7c32-3393-4cbc-8f08-fda98a85364b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2000cce93af94b9c8422852c55fc6433', 'request_id': 'req-ea4c7c32-3393-4cbc-8f08-fda98a85364b'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:34.454619 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-ea4c7c32-3393-4cbc-8f08-fda98a85364b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2000cce93af94b9c8422852c55fc6433', 'request_id': 'req-ea4c7c32-3393-4cbc-8f08-fda98a85364b'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:34.460106 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-ea4c7c32-3393-4cbc-8f08-fda98a85364b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2000cce93af94b9c8422852c55fc6433', 'request_id': 'req-ea4c7c32-3393-4cbc-8f08-fda98a85364b'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:34.472116 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-ea4c7c32-3393-4cbc-8f08-fda98a85364b admin admin] The token cache is being invalidate because project 2000cce93af94b9c8422852c55fc6433 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:48:34.472430 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-ea4c7c32-3393-4cbc-8f08-fda98a85364b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:34.473043 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-ea4c7c32-3393-4cbc-8f08-fda98a85364b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:34.474515 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1046/2081] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:48:34 2022] DELETE /identity/v3/projects/2000cce93af94b9c8422852c55fc6433 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 10:48:34.479383 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39832325-611b-4c3f-ad17-dabd572a04c9 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:34.479697 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39832325-611b-4c3f-ad17-dabd572a04c9 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:34.479962 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39832325-611b-4c3f-ad17-dabd572a04c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:34.500548 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-39832325-611b-4c3f-ad17-dabd572a04c9 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:48:34.501756 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-39832325-611b-4c3f-ad17-dabd572a04c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:34.518255 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:48:34.564299 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1036/2082] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:48:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:48:34.572071 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d38779-557b-4340-83d1-d4d00abe4feb None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:34.572355 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d38779-557b-4340-83d1-d4d00abe4feb None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:34.572634 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d38779-557b-4340-83d1-d4d00abe4feb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:34.592089 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-d8d38779-557b-4340-83d1-d4d00abe4feb None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:48:34.593391 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-d8d38779-557b-4340-83d1-d4d00abe4feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:34.601430 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1047/2083] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:48:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:48:34.607259 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39832325-611b-4c3f-ad17-dabd572a04c9 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:34.617883 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39832325-611b-4c3f-ad17-dabd572a04c9 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:34.622420 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e507c4d2-4ecb-4250-87ab-064110dcd716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:34.622945 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e507c4d2-4ecb-4250-87ab-064110dcd716 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:34.623220 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e507c4d2-4ecb-4250-87ab-064110dcd716 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:34.623522 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e507c4d2-4ecb-4250-87ab-064110dcd716 admin admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:34.624154 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e507c4d2-4ecb-4250-87ab-064110dcd716 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:48:34.624478 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e507c4d2-4ecb-4250-87ab-064110dcd716 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:34.626948 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e507c4d2-4ecb-4250-87ab-064110dcd716 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:34.638210 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1037/2084] 10.222.0.15 () {58 vars in 1153 bytes} [Sun Sep 11 10:48:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:48:34.642460 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d38779-557b-4340-83d1-d4d00abe4feb None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:34.652025 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d38779-557b-4340-83d1-d4d00abe4feb None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:34.657112 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0126cf-bafd-4ffe-92d2-1bc0915f6459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:34.657719 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0126cf-bafd-4ffe-92d2-1bc0915f6459 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:34.658001 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0126cf-bafd-4ffe-92d2-1bc0915f6459 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:34.658274 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0126cf-bafd-4ffe-92d2-1bc0915f6459 admin admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:34.658905 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0126cf-bafd-4ffe-92d2-1bc0915f6459 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:34.661551 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0126cf-bafd-4ffe-92d2-1bc0915f6459 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:34.693864 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1048/2085] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:48:34 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:48:34.695018 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e507c4d2-4ecb-4250-87ab-064110dcd716 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:34.705318 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e507c4d2-4ecb-4250-87ab-064110dcd716 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:34.762161 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a781dc1-1088-4716-ae01-cef49b2edaf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:34.762789 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a781dc1-1088-4716-ae01-cef49b2edaf1 admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:34.763094 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a781dc1-1088-4716-ae01-cef49b2edaf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:34.763415 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a781dc1-1088-4716-ae01-cef49b2edaf1 admin admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:34.764119 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a781dc1-1088-4716-ae01-cef49b2edaf1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:34.766094 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a781dc1-1088-4716-ae01-cef49b2edaf1 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:34.788297 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1038/2086] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:48:34 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:48:34.792418 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0126cf-bafd-4ffe-92d2-1bc0915f6459 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:34.803087 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0126cf-bafd-4ffe-92d2-1bc0915f6459 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:34.807836 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b624f9-0e0c-49c2-ab52-338f86824d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:34.808453 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b624f9-0e0c-49c2-ab52-338f86824d22 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:34.808736 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b624f9-0e0c-49c2-ab52-338f86824d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:34.809009 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b624f9-0e0c-49c2-ab52-338f86824d22 admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:34.809679 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b624f9-0e0c-49c2-ab52-338f86824d22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:34.811275 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b624f9-0e0c-49c2-ab52-338f86824d22 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:34.821436 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1049/2087] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:48:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 10:48:34.825449 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a781dc1-1088-4716-ae01-cef49b2edaf1 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:34.835660 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a781dc1-1088-4716-ae01-cef49b2edaf1 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:34.840406 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcc9351-b7ca-454b-87a5-13138c87cc3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:34.841009 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc9351-b7ca-454b-87a5-13138c87cc3d admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:34.841290 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc9351-b7ca-454b-87a5-13138c87cc3d admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:34.841566 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc9351-b7ca-454b-87a5-13138c87cc3d admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:34.842204 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcc9351-b7ca-454b-87a5-13138c87cc3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:34.844367 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcc9351-b7ca-454b-87a5-13138c87cc3d admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:34.855644 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1039/2088] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:48:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:48:34.863063 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b624f9-0e0c-49c2-ab52-338f86824d22 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:34.875129 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b624f9-0e0c-49c2-ab52-338f86824d22 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:34.879911 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c922bd-9f6e-4e52-ad2b-94da11f9ca75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:34.880570 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c922bd-9f6e-4e52-ad2b-94da11f9ca75 admin admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:34.880852 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c922bd-9f6e-4e52-ad2b-94da11f9ca75 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:34.881126 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c922bd-9f6e-4e52-ad2b-94da11f9ca75 admin admin] PATH_INFO: `/v3/projects/d5a897f0b39c474180185e63b0097f33/users/b4b2ad0cc0ec483b9090eff565ae2846/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:34.896045 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c922bd-9f6e-4e52-ad2b-94da11f9ca75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5a897f0b39c474180185e63b0097f33, user_id=b4b2ad0cc0ec483b9090eff565ae2846, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:34.898715 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c922bd-9f6e-4e52-ad2b-94da11f9ca75 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:34.913064 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1050/2089] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:48:34 2022] PUT /identity/v3/projects/d5a897f0b39c474180185e63b0097f33/users/b4b2ad0cc0ec483b9090eff565ae2846/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:48:34.923287 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bd2353-eca4-48c7-8568-9e2389313412 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:34.923602 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bd2353-eca4-48c7-8568-9e2389313412 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:34.923882 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bd2353-eca4-48c7-8568-9e2389313412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:34.952063 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-25bd2353-eca4-48c7-8568-9e2389313412 None None] MFA Rules not processed for user `b4b2ad0cc0ec483b9090eff565ae2846`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:48:34.953522 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-25bd2353-eca4-48c7-8568-9e2389313412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:35.020085 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1040/2090] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:48:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:48:35.024551 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c922bd-9f6e-4e52-ad2b-94da11f9ca75 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:35.047289 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:48:35.091562 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c922bd-9f6e-4e52-ad2b-94da11f9ca75 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:35.097110 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe61ea1-c3eb-47e0-8f5d-c1ec1eb43dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:35.097728 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe61ea1-c3eb-47e0-8f5d-c1ec1eb43dbd admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:35.098025 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe61ea1-c3eb-47e0-8f5d-c1ec1eb43dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:35.098821 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe61ea1-c3eb-47e0-8f5d-c1ec1eb43dbd admin admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:35.098932 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe61ea1-c3eb-47e0-8f5d-c1ec1eb43dbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:35.100839 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe61ea1-c3eb-47e0-8f5d-c1ec1eb43dbd admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:35.131905 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1051/2091] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:48:35 2022] POST /identity/v3/projects => generated 377 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 10:48:35.135950 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bd2353-eca4-48c7-8568-9e2389313412 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:35.147491 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bd2353-eca4-48c7-8568-9e2389313412 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:35.166839 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:48:35.221583 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27c79ed-84b3-4c1b-a7bc-d8b009413397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:35.222329 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27c79ed-84b3-4c1b-a7bc-d8b009413397 admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:35.222611 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27c79ed-84b3-4c1b-a7bc-d8b009413397 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:35.222885 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27c79ed-84b3-4c1b-a7bc-d8b009413397 admin admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:35.223615 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27c79ed-84b3-4c1b-a7bc-d8b009413397 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:35.225576 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27c79ed-84b3-4c1b-a7bc-d8b009413397 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:35.246884 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1041/2092] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:48:35 2022] POST /identity/v3/users => generated 289 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:48:35.251219 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe61ea1-c3eb-47e0-8f5d-c1ec1eb43dbd admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:35.261597 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe61ea1-c3eb-47e0-8f5d-c1ec1eb43dbd admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:35.267008 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647c05a2-b9cb-491e-9a98-a507b0b108d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:35.267724 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647c05a2-b9cb-491e-9a98-a507b0b108d9 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:35.268008 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647c05a2-b9cb-491e-9a98-a507b0b108d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:35.268280 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647c05a2-b9cb-491e-9a98-a507b0b108d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:35.268920 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647c05a2-b9cb-491e-9a98-a507b0b108d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:35.270516 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647c05a2-b9cb-491e-9a98-a507b0b108d9 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:35.281365 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1052/2093] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:48:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:48:35.285760 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27c79ed-84b3-4c1b-a7bc-d8b009413397 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:35.296056 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27c79ed-84b3-4c1b-a7bc-d8b009413397 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:35.300765 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c904768-1603-423c-9467-a7cc84fa73d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:35.301390 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c904768-1603-423c-9467-a7cc84fa73d6 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:35.301674 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c904768-1603-423c-9467-a7cc84fa73d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:35.301951 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c904768-1603-423c-9467-a7cc84fa73d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:35.302590 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c904768-1603-423c-9467-a7cc84fa73d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:35.304226 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c904768-1603-423c-9467-a7cc84fa73d6 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:35.314506 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1042/2094] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:48:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:48:35.318517 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647c05a2-b9cb-491e-9a98-a507b0b108d9 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:35.329205 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647c05a2-b9cb-491e-9a98-a507b0b108d9 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:35.333764 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc460ba-2f6e-4a61-af37-fa58c5e5a67a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:35.334404 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc460ba-2f6e-4a61-af37-fa58c5e5a67a admin admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:35.334682 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc460ba-2f6e-4a61-af37-fa58c5e5a67a admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:35.334951 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc460ba-2f6e-4a61-af37-fa58c5e5a67a admin admin] PATH_INFO: `/v3/projects/320868a8a64e4d6fbb82afefc2be4df0/users/dc3112b7a1b842a88b4036c46bdfefb2/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:35.349315 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc460ba-2f6e-4a61-af37-fa58c5e5a67a admin admin] RBAC: Authorizing `identity:create_grant(project_id=320868a8a64e4d6fbb82afefc2be4df0, user_id=dc3112b7a1b842a88b4036c46bdfefb2, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:35.352015 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc460ba-2f6e-4a61-af37-fa58c5e5a67a admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:35.365874 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1053/2095] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:48:35 2022] PUT /identity/v3/projects/320868a8a64e4d6fbb82afefc2be4df0/users/dc3112b7a1b842a88b4036c46bdfefb2/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:48:35.376694 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78023c14-7496-4ce8-9595-476174001ded None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:35.376983 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78023c14-7496-4ce8-9595-476174001ded None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:35.377260 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78023c14-7496-4ce8-9595-476174001ded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:35.405236 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-78023c14-7496-4ce8-9595-476174001ded None None] MFA Rules not processed for user `dc3112b7a1b842a88b4036c46bdfefb2`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:48:35.406413 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-78023c14-7496-4ce8-9595-476174001ded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:35.470793 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1043/2096] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:48:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:48:35.475220 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc460ba-2f6e-4a61-af37-fa58c5e5a67a admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:35.537902 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc460ba-2f6e-4a61-af37-fa58c5e5a67a admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:35.542534 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c317ce-a387-4b63-a937-dfe3a51decec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:35.543166 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c317ce-a387-4b63-a937-dfe3a51decec admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:35.543471 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c317ce-a387-4b63-a937-dfe3a51decec admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:35.543747 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c317ce-a387-4b63-a937-dfe3a51decec admin admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:35.544368 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c317ce-a387-4b63-a937-dfe3a51decec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:35.546265 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c317ce-a387-4b63-a937-dfe3a51decec admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:35.572505 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1054/2097] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:48:35 2022] POST /identity/v3/projects => generated 375 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:48:35.576616 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78023c14-7496-4ce8-9595-476174001ded None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:35.587265 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78023c14-7496-4ce8-9595-476174001ded None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:35.642568 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af705c7-614a-4807-8111-6f5ecbfa8633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:35.643172 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af705c7-614a-4807-8111-6f5ecbfa8633 admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:35.643479 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af705c7-614a-4807-8111-6f5ecbfa8633 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:35.643758 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af705c7-614a-4807-8111-6f5ecbfa8633 admin admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:35.644383 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af705c7-614a-4807-8111-6f5ecbfa8633 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:35.646355 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af705c7-614a-4807-8111-6f5ecbfa8633 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:35.668023 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1044/2098] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:48:35 2022] POST /identity/v3/users => generated 294 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:48:35.672675 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c317ce-a387-4b63-a937-dfe3a51decec admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:35.682917 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c317ce-a387-4b63-a937-dfe3a51decec admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:35.687750 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74876e29-352e-4f6f-8169-3e7ef5874ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:35.688347 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74876e29-352e-4f6f-8169-3e7ef5874ad2 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:35.688627 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74876e29-352e-4f6f-8169-3e7ef5874ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:35.688901 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74876e29-352e-4f6f-8169-3e7ef5874ad2 admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:35.689536 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74876e29-352e-4f6f-8169-3e7ef5874ad2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:35.691130 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74876e29-352e-4f6f-8169-3e7ef5874ad2 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:35.701883 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1055/2099] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:48:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:48:35.706409 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af705c7-614a-4807-8111-6f5ecbfa8633 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:35.716830 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af705c7-614a-4807-8111-6f5ecbfa8633 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:35.722419 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d27f38-7e91-47a2-8b7d-006c66e582d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:35.723208 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d27f38-7e91-47a2-8b7d-006c66e582d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:35.723524 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d27f38-7e91-47a2-8b7d-006c66e582d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:35.723796 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d27f38-7e91-47a2-8b7d-006c66e582d7 admin admin] PATH_INFO: `/v3/projects/dab2b35616fc41ff8aeb4b534cc04bd1/users/ea6d7200c6904c37b906bc183e6f8e70/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:35.736980 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:48:35.740137 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d27f38-7e91-47a2-8b7d-006c66e582d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dab2b35616fc41ff8aeb4b534cc04bd1, user_id=ea6d7200c6904c37b906bc183e6f8e70, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:35.742811 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d27f38-7e91-47a2-8b7d-006c66e582d7 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:35.756632 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1045/2100] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:48:35 2022] PUT /identity/v3/projects/dab2b35616fc41ff8aeb4b534cc04bd1/users/ea6d7200c6904c37b906bc183e6f8e70/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:48:35.760588 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74876e29-352e-4f6f-8169-3e7ef5874ad2 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:35.771959 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74876e29-352e-4f6f-8169-3e7ef5874ad2 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:35.813617 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:48:35.829159 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e06c0f2-bbe3-4304-9f44-258158913081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:35.829770 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e06c0f2-bbe3-4304-9f44-258158913081 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:35.830050 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e06c0f2-bbe3-4304-9f44-258158913081 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:35.830324 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e06c0f2-bbe3-4304-9f44-258158913081 admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:35.830955 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e06c0f2-bbe3-4304-9f44-258158913081 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:35.832581 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e06c0f2-bbe3-4304-9f44-258158913081 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:35.842072 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1056/2101] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:48:35 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:48:35.847065 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d27f38-7e91-47a2-8b7d-006c66e582d7 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:35.857471 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d27f38-7e91-47a2-8b7d-006c66e582d7 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:35.862678 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74bb942-b267-4f44-a19b-43ee4087c790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:35.863319 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74bb942-b267-4f44-a19b-43ee4087c790 admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:35.863649 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74bb942-b267-4f44-a19b-43ee4087c790 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:35.863922 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74bb942-b267-4f44-a19b-43ee4087c790 admin admin] PATH_INFO: `/v3/projects/dab2b35616fc41ff8aeb4b534cc04bd1/users/ea6d7200c6904c37b906bc183e6f8e70/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:35.865823 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74bb942-b267-4f44-a19b-43ee4087c790 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dab2b35616fc41ff8aeb4b534cc04bd1, user_id=ea6d7200c6904c37b906bc183e6f8e70, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:35.868489 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74bb942-b267-4f44-a19b-43ee4087c790 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:35.878395 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1046/2102] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:48:35 2022] PUT /identity/v3/projects/dab2b35616fc41ff8aeb4b534cc04bd1/users/ea6d7200c6904c37b906bc183e6f8e70/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:48:35.889529 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd684a-9b08-4b12-8428-d822c0e453ae None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:35.889825 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd684a-9b08-4b12-8428-d822c0e453ae None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:35.890685 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd684a-9b08-4b12-8428-d822c0e453ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:35.919056 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-cfbd684a-9b08-4b12-8428-d822c0e453ae None None] MFA Rules not processed for user `ea6d7200c6904c37b906bc183e6f8e70`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:48:35.920221 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-cfbd684a-9b08-4b12-8428-d822c0e453ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:35.990166 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1057/2103] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:48:35 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 10:48:36.002628 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74bb942-b267-4f44-a19b-43ee4087c790 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:36.004402 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-d74bb942-b267-4f44-a19b-43ee4087c790 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:36.065349 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74bb942-b267-4f44-a19b-43ee4087c790 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:36.069092 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a7a9ce-3691-4a47-b1c6-b308baf73462 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:36.069709 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a7a9ce-3691-4a47-b1c6-b308baf73462 service cinder] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:36.069991 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a7a9ce-3691-4a47-b1c6-b308baf73462 service cinder] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:36.070264 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a7a9ce-3691-4a47-b1c6-b308baf73462 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:36.131544 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a7a9ce-3691-4a47-b1c6-b308baf73462 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:36.132973 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a7a9ce-3691-4a47-b1c6-b308baf73462 service cinder] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:36.142299 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1047/2104] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:48:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:48:39.804206 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbd684a-9b08-4b12-8428-d822c0e453ae None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:39.805701 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-cfbd684a-9b08-4b12-8428-d822c0e453ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:39.818081 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbd684a-9b08-4b12-8428-d822c0e453ae None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:39.821780 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eea861a-716c-4c38-9fef-635ff74ee612 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:39.822422 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eea861a-716c-4c38-9fef-635ff74ee612 service cinder] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:39.822821 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eea861a-716c-4c38-9fef-635ff74ee612 service cinder] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:39.823098 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eea861a-716c-4c38-9fef-635ff74ee612 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:39.903049 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eea861a-716c-4c38-9fef-635ff74ee612 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:39.904579 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eea861a-716c-4c38-9fef-635ff74ee612 service cinder] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:39.916570 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1058/2105] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:48:39 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:48:40.476338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a7a9ce-3691-4a47-b1c6-b308baf73462 service cinder] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:40.486046 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a7a9ce-3691-4a47-b1c6-b308baf73462 service cinder] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:40.489872 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ba0c6c-18f0-4a63-bf38-19613761c69f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:40.490674 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba0c6c-18f0-4a63-bf38-19613761c69f service cinder] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:40.490951 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba0c6c-18f0-4a63-bf38-19613761c69f service cinder] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:40.491291 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba0c6c-18f0-4a63-bf38-19613761c69f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:40.501193 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ba0c6c-18f0-4a63-bf38-19613761c69f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:40.502595 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ba0c6c-18f0-4a63-bf38-19613761c69f service cinder] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:40.511891 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1048/2106] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:48:40 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:48:45.618165 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eea861a-716c-4c38-9fef-635ff74ee612 service cinder] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:45.764609 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eea861a-716c-4c38-9fef-635ff74ee612 service cinder] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:45.773781 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b52e818-b9cc-46ce-9c9e-e3714bf603cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:45.775136 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b52e818-b9cc-46ce-9c9e-e3714bf603cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:45.775607 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b52e818-b9cc-46ce-9c9e-e3714bf603cc admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:45.776526 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b52e818-b9cc-46ce-9c9e-e3714bf603cc admin admin] PATH_INFO: `/v3/users/b4b2ad0cc0ec483b9090eff565ae2846` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:45.778287 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b52e818-b9cc-46ce-9c9e-e3714bf603cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4b2ad0cc0ec483b9090eff565ae2846)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:45.780481 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b52e818-b9cc-46ce-9c9e-e3714bf603cc admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:45.787550 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:48:45.846064 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-2b52e818-b9cc-46ce-9c9e-e3714bf603cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4b2ad0cc0ec483b9090eff565ae2846'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:45.853618 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-2b52e818-b9cc-46ce-9c9e-e3714bf603cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4b2ad0cc0ec483b9090eff565ae2846'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:45.854375 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-2b52e818-b9cc-46ce-9c9e-e3714bf603cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4b2ad0cc0ec483b9090eff565ae2846'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:45.854983 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-2b52e818-b9cc-46ce-9c9e-e3714bf603cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4b2ad0cc0ec483b9090eff565ae2846'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:45.868534 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-2b52e818-b9cc-46ce-9c9e-e3714bf603cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4b2ad0cc0ec483b9090eff565ae2846'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:45.881360 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1059/2107] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:48:45 2022] DELETE /identity/v3/users/b4b2ad0cc0ec483b9090eff565ae2846 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:48:45.886073 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658371a-8197-4f73-bbf1-8ab4a4947f68 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:45.886549 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658371a-8197-4f73-bbf1-8ab4a4947f68 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:45.886645 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658371a-8197-4f73-bbf1-8ab4a4947f68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:45.905685 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-e658371a-8197-4f73-bbf1-8ab4a4947f68 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:48:45.907496 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-e658371a-8197-4f73-bbf1-8ab4a4947f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:45.976931 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1049/2108] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:48:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:48:45.990812 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b52e818-b9cc-46ce-9c9e-e3714bf603cc admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:45.990812 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-2b52e818-b9cc-46ce-9c9e-e3714bf603cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:46.019654 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b93d27f-c331-4cc4-8f82-bef50f416521 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:46.020429 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b93d27f-c331-4cc4-8f82-bef50f416521 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:46.021401 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b93d27f-c331-4cc4-8f82-bef50f416521 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:46.023142 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1050/2109] 10.222.0.15 () {54 vars in 921 bytes} [Sun Sep 11 10:48:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 10:48:46.037995 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb74630-8056-4f52-a3f4-8898eee3ddea None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:46.038526 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb74630-8056-4f52-a3f4-8898eee3ddea None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:46.039376 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb74630-8056-4f52-a3f4-8898eee3ddea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:46.052600 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b52e818-b9cc-46ce-9c9e-e3714bf603cc admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:46.056622 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7a6084-9d76-4311-8c18-394bd113b008 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:46.057247 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7a6084-9d76-4311-8c18-394bd113b008 service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:46.057529 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7a6084-9d76-4311-8c18-394bd113b008 service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:46.057806 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7a6084-9d76-4311-8c18-394bd113b008 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:46.062974 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-bcb74630-8056-4f52-a3f4-8898eee3ddea None None] MFA Rules not processed for user `08477b4c88f04c9e97391aba401879e0`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:48:46.064919 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-bcb74630-8056-4f52-a3f4-8898eee3ddea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:46.068283 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7a6084-9d76-4311-8c18-394bd113b008 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:46.069756 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7a6084-9d76-4311-8c18-394bd113b008 service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:46.081187 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1060/2110] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:48:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:48:46.084926 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:48:46.133895 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7a6084-9d76-4311-8c18-394bd113b008 service neutron] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:46.135396 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-0e7a6084-9d76-4311-8c18-394bd113b008 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:46.150143 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7a6084-9d76-4311-8c18-394bd113b008 service neutron] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:46.156443 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868ba981-64b4-4656-9fa6-eb3e77407bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:46.157208 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ba981-64b4-4656-9fa6-eb3e77407bd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:46.157502 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ba981-64b4-4656-9fa6-eb3e77407bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:46.158160 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ba981-64b4-4656-9fa6-eb3e77407bd6 admin admin] PATH_INFO: `/v3/users/ea9f58a33d1449099bd39f8d09f9f0cd` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:46.159132 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868ba981-64b4-4656-9fa6-eb3e77407bd6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea9f58a33d1449099bd39f8d09f9f0cd)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:46.161681 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868ba981-64b4-4656-9fa6-eb3e77407bd6 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:46.172735 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1051/2111] 10.222.0.15 () {58 vars in 1018 bytes} [Sun Sep 11 10:48:46 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 10:48:46.187186 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb74630-8056-4f52-a3f4-8898eee3ddea None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:46.193932 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-bcb74630-8056-4f52-a3f4-8898eee3ddea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:46.208477 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb74630-8056-4f52-a3f4-8898eee3ddea None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:46.215144 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312a1206-db86-4e8e-a35e-545a085ab480 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08477b4c88f04c9e97391aba401879e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:46.216354 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a1206-db86-4e8e-a35e-545a085ab480 service placement] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:46.216648 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a1206-db86-4e8e-a35e-545a085ab480 service placement] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:46.216922 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a1206-db86-4e8e-a35e-545a085ab480 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:46.219574 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-868ba981-64b4-4656-9fa6-eb3e77407bd6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea9f58a33d1449099bd39f8d09f9f0cd'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:46.227565 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312a1206-db86-4e8e-a35e-545a085ab480 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:46.228528 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-868ba981-64b4-4656-9fa6-eb3e77407bd6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea9f58a33d1449099bd39f8d09f9f0cd'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:46.229347 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-868ba981-64b4-4656-9fa6-eb3e77407bd6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea9f58a33d1449099bd39f8d09f9f0cd'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:46.229476 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312a1206-db86-4e8e-a35e-545a085ab480 service placement] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:46.229990 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-868ba981-64b4-4656-9fa6-eb3e77407bd6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea9f58a33d1449099bd39f8d09f9f0cd'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:46.240296 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1052/2112] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:48:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 10:48:46.247246 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-868ba981-64b4-4656-9fa6-eb3e77407bd6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea9f58a33d1449099bd39f8d09f9f0cd'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:46.260128 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:48:46.268255 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1061/2113] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:48:46 2022] DELETE /identity/v3/users/ea9f58a33d1449099bd39f8d09f9f0cd => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:48:46.404313 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312a1206-db86-4e8e-a35e-545a085ab480 service placement] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:46.406977 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-312a1206-db86-4e8e-a35e-545a085ab480 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:46.438723 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312a1206-db86-4e8e-a35e-545a085ab480 service placement] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:46.470131 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868ba981-64b4-4656-9fa6-eb3e77407bd6 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:46.471570 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-868ba981-64b4-4656-9fa6-eb3e77407bd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:46.513598 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2dc8e4-5133-467e-94a1-29931e90b608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:46.514233 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2dc8e4-5133-467e-94a1-29931e90b608 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:46.514516 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2dc8e4-5133-467e-94a1-29931e90b608 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:46.514821 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2dc8e4-5133-467e-94a1-29931e90b608 admin admin] PATH_INFO: `/v3/projects/d5a897f0b39c474180185e63b0097f33` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:46.516248 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2dc8e4-5133-467e-94a1-29931e90b608 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5a897f0b39c474180185e63b0097f33)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:46.518143 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2dc8e4-5133-467e-94a1-29931e90b608 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:46.543225 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868ba981-64b4-4656-9fa6-eb3e77407bd6 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:46.548937 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef1b81a-39c8-405f-98b6-f3ec5140f1f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:46.549564 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1b81a-39c8-405f-98b6-f3ec5140f1f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:46.550208 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1b81a-39c8-405f-98b6-f3ec5140f1f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:46.550208 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1b81a-39c8-405f-98b6-f3ec5140f1f7 admin admin] PATH_INFO: `/v3/projects/74e895a982b34e958af26f5287567949` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:46.551219 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef1b81a-39c8-405f-98b6-f3ec5140f1f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74e895a982b34e958af26f5287567949)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:46.553166 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef1b81a-39c8-405f-98b6-f3ec5140f1f7 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:46.566005 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-7b2dc8e4-5133-467e-94a1-29931e90b608 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5a897f0b39c474180185e63b0097f33', 'request_id': 'req-7b2dc8e4-5133-467e-94a1-29931e90b608'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:46.572792 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-7b2dc8e4-5133-467e-94a1-29931e90b608 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5a897f0b39c474180185e63b0097f33', 'request_id': 'req-7b2dc8e4-5133-467e-94a1-29931e90b608'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:46.582173 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-7b2dc8e4-5133-467e-94a1-29931e90b608 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5a897f0b39c474180185e63b0097f33', 'request_id': 'req-7b2dc8e4-5133-467e-94a1-29931e90b608'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:46.592978 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-7b2dc8e4-5133-467e-94a1-29931e90b608 admin admin] The token cache is being invalidate because project d5a897f0b39c474180185e63b0097f33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90653) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:48:46.593417 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-7b2dc8e4-5133-467e-94a1-29931e90b608 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:46.594132 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-7b2dc8e4-5133-467e-94a1-29931e90b608 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:46.595914 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1053/2114] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:48:46 2022] DELETE /identity/v3/projects/d5a897f0b39c474180185e63b0097f33 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:48:46.599554 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2dc8e4-5133-467e-94a1-29931e90b608 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:46.600909 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-7b2dc8e4-5133-467e-94a1-29931e90b608 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:46.601468 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-cef1b81a-39c8-405f-98b6-f3ec5140f1f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74e895a982b34e958af26f5287567949', 'request_id': 'req-cef1b81a-39c8-405f-98b6-f3ec5140f1f7'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:46.608119 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-cef1b81a-39c8-405f-98b6-f3ec5140f1f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74e895a982b34e958af26f5287567949', 'request_id': 'req-cef1b81a-39c8-405f-98b6-f3ec5140f1f7'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:46.614188 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-cef1b81a-39c8-405f-98b6-f3ec5140f1f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74e895a982b34e958af26f5287567949', 'request_id': 'req-cef1b81a-39c8-405f-98b6-f3ec5140f1f7'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:46.627717 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-cef1b81a-39c8-405f-98b6-f3ec5140f1f7 admin admin] The token cache is being invalidate because project 74e895a982b34e958af26f5287567949 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:48:46.628337 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-cef1b81a-39c8-405f-98b6-f3ec5140f1f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:46.628667 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-cef1b81a-39c8-405f-98b6-f3ec5140f1f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:46.630387 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1062/2115] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:48:46 2022] DELETE /identity/v3/projects/74e895a982b34e958af26f5287567949 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:48:46.634779 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda1ea4c-7335-4e82-bdf1-d0c6d59fa059 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:46.635061 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda1ea4c-7335-4e82-bdf1-d0c6d59fa059 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:46.635341 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda1ea4c-7335-4e82-bdf1-d0c6d59fa059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:46.637596 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:48:46.655707 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-cda1ea4c-7335-4e82-bdf1-d0c6d59fa059 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:48:46.656900 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-cda1ea4c-7335-4e82-bdf1-d0c6d59fa059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:46.669456 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2dc8e4-5133-467e-94a1-29931e90b608 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:46.717897 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1063/2116] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:48:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 10:48:46.725452 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae10a88-0865-4398-8058-9258aea45066 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:46.725740 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae10a88-0865-4398-8058-9258aea45066 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:46.726016 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae10a88-0865-4398-8058-9258aea45066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:46.726774 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab93ec4-807d-4d21-aba3-6bf6eb9667c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:46.727395 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab93ec4-807d-4d21-aba3-6bf6eb9667c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:46.727707 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab93ec4-807d-4d21-aba3-6bf6eb9667c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:46.727977 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab93ec4-807d-4d21-aba3-6bf6eb9667c9 admin admin] PATH_INFO: `/v3/users/dc3112b7a1b842a88b4036c46bdfefb2` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:46.729094 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab93ec4-807d-4d21-aba3-6bf6eb9667c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc3112b7a1b842a88b4036c46bdfefb2)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:46.730997 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab93ec4-807d-4d21-aba3-6bf6eb9667c9 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:46.746573 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-eae10a88-0865-4398-8058-9258aea45066 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:48:46.748042 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-eae10a88-0865-4398-8058-9258aea45066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:46.758633 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1064/2117] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:48:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 10:48:46.762862 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae10a88-0865-4398-8058-9258aea45066 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:46.775979 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae10a88-0865-4398-8058-9258aea45066 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:46.782672 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee3578a-6ddb-491b-8b47-622f379daff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:46.783379 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee3578a-6ddb-491b-8b47-622f379daff1 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:46.783719 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee3578a-6ddb-491b-8b47-622f379daff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:46.783962 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee3578a-6ddb-491b-8b47-622f379daff1 admin admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:46.784666 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee3578a-6ddb-491b-8b47-622f379daff1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:48:46.785067 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee3578a-6ddb-491b-8b47-622f379daff1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:46.786923 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee3578a-6ddb-491b-8b47-622f379daff1 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:46.790673 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-dab93ec4-807d-4d21-aba3-6bf6eb9667c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc3112b7a1b842a88b4036c46bdfefb2'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:46.800206 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-dab93ec4-807d-4d21-aba3-6bf6eb9667c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc3112b7a1b842a88b4036c46bdfefb2'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:46.800709 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-dab93ec4-807d-4d21-aba3-6bf6eb9667c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc3112b7a1b842a88b4036c46bdfefb2'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:46.812180 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1065/2118] 10.222.0.15 () {58 vars in 1153 bytes} [Sun Sep 11 10:48:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:48:46.812279 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-dab93ec4-807d-4d21-aba3-6bf6eb9667c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc3112b7a1b842a88b4036c46bdfefb2'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:46.812357 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee3578a-6ddb-491b-8b47-622f379daff1 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:46.812357 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-3ee3578a-6ddb-491b-8b47-622f379daff1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:46.812537 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-dab93ec4-807d-4d21-aba3-6bf6eb9667c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc3112b7a1b842a88b4036c46bdfefb2'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:46.826428 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee3578a-6ddb-491b-8b47-622f379daff1 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:46.831848 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1054/2119] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:48:46 2022] DELETE /identity/v3/users/dc3112b7a1b842a88b4036c46bdfefb2 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 10:48:46.832673 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48b84e2-e5fa-426f-8206-fd0efefd55d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:46.833596 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b84e2-e5fa-426f-8206-fd0efefd55d5 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:46.833596 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b84e2-e5fa-426f-8206-fd0efefd55d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:46.833833 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b84e2-e5fa-426f-8206-fd0efefd55d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:46.834453 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48b84e2-e5fa-426f-8206-fd0efefd55d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:46.836567 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48b84e2-e5fa-426f-8206-fd0efefd55d5 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:46.843559 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:48:46.873525 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1066/2120] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:48:46 2022] POST /identity/v3/projects => generated 371 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 10:48:46.875548 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab93ec4-807d-4d21-aba3-6bf6eb9667c9 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:46.947013 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab93ec4-807d-4d21-aba3-6bf6eb9667c9 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:46.950761 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:48:47.020984 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bdeeb2-df3b-4090-bd38-4d01bb6279bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:47.021607 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bdeeb2-df3b-4090-bd38-4d01bb6279bd admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:47.021886 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bdeeb2-df3b-4090-bd38-4d01bb6279bd admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:47.022155 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bdeeb2-df3b-4090-bd38-4d01bb6279bd admin admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:47.022780 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bdeeb2-df3b-4090-bd38-4d01bb6279bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:47.024724 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bdeeb2-df3b-4090-bd38-4d01bb6279bd admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:47.051003 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1055/2121] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:48:46 2022] POST /identity/v3/users => generated 286 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:48:47.052503 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48b84e2-e5fa-426f-8206-fd0efefd55d5 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:47.063836 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48b84e2-e5fa-426f-8206-fd0efefd55d5 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:47.068622 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe20f0b-5c9f-4953-9a8f-5217ded045b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:47.069230 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe20f0b-5c9f-4953-9a8f-5217ded045b9 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:47.069507 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe20f0b-5c9f-4953-9a8f-5217ded045b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:47.069809 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe20f0b-5c9f-4953-9a8f-5217ded045b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:47.070449 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe20f0b-5c9f-4953-9a8f-5217ded045b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:47.072098 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe20f0b-5c9f-4953-9a8f-5217ded045b9 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:47.083522 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1067/2122] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:48:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:48:47.087554 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bdeeb2-df3b-4090-bd38-4d01bb6279bd admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:47.098429 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bdeeb2-df3b-4090-bd38-4d01bb6279bd admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:47.103312 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54668d23-6f9a-45be-bbd8-1a61f299269a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:47.103937 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54668d23-6f9a-45be-bbd8-1a61f299269a admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:47.104265 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54668d23-6f9a-45be-bbd8-1a61f299269a admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:47.104488 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54668d23-6f9a-45be-bbd8-1a61f299269a admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:47.105210 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54668d23-6f9a-45be-bbd8-1a61f299269a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:47.106876 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54668d23-6f9a-45be-bbd8-1a61f299269a admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:47.115589 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe20f0b-5c9f-4953-9a8f-5217ded045b9 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:47.117529 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-fbe20f0b-5c9f-4953-9a8f-5217ded045b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:47.118050 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1056/2123] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:48:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:48:47.122134 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54668d23-6f9a-45be-bbd8-1a61f299269a admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:47.133772 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54668d23-6f9a-45be-bbd8-1a61f299269a admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:47.136119 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe20f0b-5c9f-4953-9a8f-5217ded045b9 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:47.140453 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55794885-6dbe-4063-a697-61fb58c9094a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:47.141156 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55794885-6dbe-4063-a697-61fb58c9094a admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:47.141385 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55794885-6dbe-4063-a697-61fb58c9094a admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:47.141664 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55794885-6dbe-4063-a697-61fb58c9094a admin admin] PATH_INFO: `/v3/projects/52916d27e2e84d9c898c2084cdbdbb42/users/8ebd403c3d07406f82d0163b7bb0bd99/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:47.142626 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbac28c4-3e1f-428a-9068-4981020353ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:47.143594 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbac28c4-3e1f-428a-9068-4981020353ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:47.144158 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbac28c4-3e1f-428a-9068-4981020353ad admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:47.144465 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbac28c4-3e1f-428a-9068-4981020353ad admin admin] PATH_INFO: `/v3/projects/320868a8a64e4d6fbb82afefc2be4df0` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:47.146020 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbac28c4-3e1f-428a-9068-4981020353ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=320868a8a64e4d6fbb82afefc2be4df0)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:47.148412 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbac28c4-3e1f-428a-9068-4981020353ad admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:47.160177 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55794885-6dbe-4063-a697-61fb58c9094a admin admin] RBAC: Authorizing `identity:create_grant(project_id=52916d27e2e84d9c898c2084cdbdbb42, user_id=8ebd403c3d07406f82d0163b7bb0bd99, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:47.163096 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55794885-6dbe-4063-a697-61fb58c9094a admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:47.179219 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1057/2124] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:48:47 2022] PUT /identity/v3/projects/52916d27e2e84d9c898c2084cdbdbb42/users/8ebd403c3d07406f82d0163b7bb0bd99/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:48:47.191662 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c263ef-4552-4b69-859d-d4e70aa5cd89 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:47.192100 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c263ef-4552-4b69-859d-d4e70aa5cd89 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:47.192492 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c263ef-4552-4b69-859d-d4e70aa5cd89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:47.211042 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-cbac28c4-3e1f-428a-9068-4981020353ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '320868a8a64e4d6fbb82afefc2be4df0', 'request_id': 'req-cbac28c4-3e1f-428a-9068-4981020353ad'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:47.219121 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-cbac28c4-3e1f-428a-9068-4981020353ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '320868a8a64e4d6fbb82afefc2be4df0', 'request_id': 'req-cbac28c4-3e1f-428a-9068-4981020353ad'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:47.224142 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-c2c263ef-4552-4b69-859d-d4e70aa5cd89 None None] MFA Rules not processed for user `8ebd403c3d07406f82d0163b7bb0bd99`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:48:47.225361 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-c2c263ef-4552-4b69-859d-d4e70aa5cd89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:47.225617 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-cbac28c4-3e1f-428a-9068-4981020353ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '320868a8a64e4d6fbb82afefc2be4df0', 'request_id': 'req-cbac28c4-3e1f-428a-9068-4981020353ad'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:47.241913 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-cbac28c4-3e1f-428a-9068-4981020353ad admin admin] The token cache is being invalidate because project 320868a8a64e4d6fbb82afefc2be4df0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:48:47.242531 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-cbac28c4-3e1f-428a-9068-4981020353ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:47.243417 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-cbac28c4-3e1f-428a-9068-4981020353ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:47.245995 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1068/2125] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:48:47 2022] DELETE /identity/v3/projects/320868a8a64e4d6fbb82afefc2be4df0 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 10:48:47.250347 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbac28c4-3e1f-428a-9068-4981020353ad admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:47.251801 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-cbac28c4-3e1f-428a-9068-4981020353ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:47.308308 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1058/2126] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:48:47 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 10:48:47.314355 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c263ef-4552-4b69-859d-d4e70aa5cd89 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:47.315911 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-c2c263ef-4552-4b69-859d-d4e70aa5cd89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:47.331767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbac28c4-3e1f-428a-9068-4981020353ad admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:47.336908 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08d2db1-3155-4f6e-b057-8812cbade656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:47.337574 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08d2db1-3155-4f6e-b057-8812cbade656 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:47.337857 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08d2db1-3155-4f6e-b057-8812cbade656 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:47.338170 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08d2db1-3155-4f6e-b057-8812cbade656 admin admin] PATH_INFO: `/v3/users/ea6d7200c6904c37b906bc183e6f8e70` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:47.339354 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08d2db1-3155-4f6e-b057-8812cbade656 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea6d7200c6904c37b906bc183e6f8e70)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:47.341363 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08d2db1-3155-4f6e-b057-8812cbade656 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:47.379019 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:48:47.395673 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c263ef-4552-4b69-859d-d4e70aa5cd89 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:47.395820 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-d08d2db1-3155-4f6e-b057-8812cbade656 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea6d7200c6904c37b906bc183e6f8e70'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:47.401019 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c237885-95a7-466a-b9d7-5c5cd94e1531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:47.401626 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c237885-95a7-466a-b9d7-5c5cd94e1531 admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:47.401967 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c237885-95a7-466a-b9d7-5c5cd94e1531 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:47.402180 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c237885-95a7-466a-b9d7-5c5cd94e1531 admin admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:47.403050 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c237885-95a7-466a-b9d7-5c5cd94e1531 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:47.404977 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c237885-95a7-466a-b9d7-5c5cd94e1531 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:47.405441 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-d08d2db1-3155-4f6e-b057-8812cbade656 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea6d7200c6904c37b906bc183e6f8e70'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:47.406281 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-d08d2db1-3155-4f6e-b057-8812cbade656 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea6d7200c6904c37b906bc183e6f8e70'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:47.406919 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-d08d2db1-3155-4f6e-b057-8812cbade656 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea6d7200c6904c37b906bc183e6f8e70'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:47.420148 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-d08d2db1-3155-4f6e-b057-8812cbade656 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea6d7200c6904c37b906bc183e6f8e70'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:47.428123 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=81,keystone:INSERT=3,keystone:DELETE=16 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:48:47.442717 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1059/2128] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:48:47 2022] POST /identity/v3/projects => generated 371 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:48:47.442858 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1069/2128] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:48:47 2022] DELETE /identity/v3/users/ea6d7200c6904c37b906bc183e6f8e70 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:48:47.443887 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c237885-95a7-466a-b9d7-5c5cd94e1531 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:47.445578 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-2c237885-95a7-466a-b9d7-5c5cd94e1531 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:47.521964 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c237885-95a7-466a-b9d7-5c5cd94e1531 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:47.614108 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a366ae3-1369-4de9-9ecc-1c2f58820834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:47.614722 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a366ae3-1369-4de9-9ecc-1c2f58820834 admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:47.615000 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a366ae3-1369-4de9-9ecc-1c2f58820834 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:47.615268 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a366ae3-1369-4de9-9ecc-1c2f58820834 admin admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:47.615990 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a366ae3-1369-4de9-9ecc-1c2f58820834 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:47.617969 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a366ae3-1369-4de9-9ecc-1c2f58820834 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:47.647173 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1060/2129] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:48:47 2022] POST /identity/v3/users => generated 292 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:48:47.648468 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08d2db1-3155-4f6e-b057-8812cbade656 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:47.658751 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08d2db1-3155-4f6e-b057-8812cbade656 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:47.663810 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72432a9-cab7-4479-9322-b59229901b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:47.664449 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72432a9-cab7-4479-9322-b59229901b36 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:47.664783 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72432a9-cab7-4479-9322-b59229901b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:47.665026 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72432a9-cab7-4479-9322-b59229901b36 admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:47.665697 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72432a9-cab7-4479-9322-b59229901b36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:47.667702 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72432a9-cab7-4479-9322-b59229901b36 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:47.682389 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1070/2130] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:48:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:48:47.683784 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a366ae3-1369-4de9-9ecc-1c2f58820834 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:47.695386 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a366ae3-1369-4de9-9ecc-1c2f58820834 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:47.700912 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79da456e-8caf-45ec-9fb9-334cdaa8bf6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:47.701568 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da456e-8caf-45ec-9fb9-334cdaa8bf6b admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:47.701847 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da456e-8caf-45ec-9fb9-334cdaa8bf6b admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:47.702118 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da456e-8caf-45ec-9fb9-334cdaa8bf6b admin admin] PATH_INFO: `/v3/projects/df6b2c180247434f87eb7914685682c0/users/84b8e4ab226747cb9bba2a31a54c0d81/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:47.727996 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79da456e-8caf-45ec-9fb9-334cdaa8bf6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=df6b2c180247434f87eb7914685682c0, user_id=84b8e4ab226747cb9bba2a31a54c0d81, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:47.730694 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79da456e-8caf-45ec-9fb9-334cdaa8bf6b admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:47.741730 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72432a9-cab7-4479-9322-b59229901b36 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:47.745020 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-c72432a9-cab7-4479-9322-b59229901b36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:47.753171 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1061/2131] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:48:47 2022] PUT /identity/v3/projects/df6b2c180247434f87eb7914685682c0/users/84b8e4ab226747cb9bba2a31a54c0d81/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:48:47.756957 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79da456e-8caf-45ec-9fb9-334cdaa8bf6b admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:47.771988 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72432a9-cab7-4479-9322-b59229901b36 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:47.825636 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:48:47.843565 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79da456e-8caf-45ec-9fb9-334cdaa8bf6b admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:47.855207 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc60ac2-dd63-4948-87a2-7066794973f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:47.855900 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc60ac2-dd63-4948-87a2-7066794973f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:47.856182 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc60ac2-dd63-4948-87a2-7066794973f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:47.857273 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc60ac2-dd63-4948-87a2-7066794973f5 admin admin] PATH_INFO: `/v3/projects/dab2b35616fc41ff8aeb4b534cc04bd1` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:47.858395 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc60ac2-dd63-4948-87a2-7066794973f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dab2b35616fc41ff8aeb4b534cc04bd1)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:47.859193 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd65406e-b058-4ac5-8c9f-d94d7a036b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:47.859869 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd65406e-b058-4ac5-8c9f-d94d7a036b0d admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:47.860187 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd65406e-b058-4ac5-8c9f-d94d7a036b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:47.860423 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd65406e-b058-4ac5-8c9f-d94d7a036b0d admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:47.860642 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc60ac2-dd63-4948-87a2-7066794973f5 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:47.861114 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd65406e-b058-4ac5-8c9f-d94d7a036b0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:47.862763 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd65406e-b058-4ac5-8c9f-d94d7a036b0d admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:47.875909 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1062/2132] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:48:47 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:48:47.879887 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd65406e-b058-4ac5-8c9f-d94d7a036b0d admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:47.893314 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd65406e-b058-4ac5-8c9f-d94d7a036b0d admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:47.899709 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0b5352-d687-4719-9f08-5533d1799515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:47.900370 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0b5352-d687-4719-9f08-5533d1799515 admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:47.900663 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0b5352-d687-4719-9f08-5533d1799515 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:47.900952 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0b5352-d687-4719-9f08-5533d1799515 admin admin] PATH_INFO: `/v3/projects/df6b2c180247434f87eb7914685682c0/users/84b8e4ab226747cb9bba2a31a54c0d81/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:47.902981 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0b5352-d687-4719-9f08-5533d1799515 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df6b2c180247434f87eb7914685682c0, user_id=84b8e4ab226747cb9bba2a31a54c0d81, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:47.905783 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0b5352-d687-4719-9f08-5533d1799515 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:47.917277 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1063/2133] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:48:47 2022] PUT /identity/v3/projects/df6b2c180247434f87eb7914685682c0/users/84b8e4ab226747cb9bba2a31a54c0d81/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 10:48:47.929491 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-4fc60ac2-dd63-4948-87a2-7066794973f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dab2b35616fc41ff8aeb4b534cc04bd1', 'request_id': 'req-4fc60ac2-dd63-4948-87a2-7066794973f5'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:47.933146 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dadc62-1963-4695-98a0-33fff20b062b None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:47.933439 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dadc62-1963-4695-98a0-33fff20b062b None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:47.933714 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dadc62-1963-4695-98a0-33fff20b062b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:47.937490 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-4fc60ac2-dd63-4948-87a2-7066794973f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dab2b35616fc41ff8aeb4b534cc04bd1', 'request_id': 'req-4fc60ac2-dd63-4948-87a2-7066794973f5'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:47.943525 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-4fc60ac2-dd63-4948-87a2-7066794973f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dab2b35616fc41ff8aeb4b534cc04bd1', 'request_id': 'req-4fc60ac2-dd63-4948-87a2-7066794973f5'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:47.956295 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-4fc60ac2-dd63-4948-87a2-7066794973f5 admin admin] The token cache is being invalidate because project dab2b35616fc41ff8aeb4b534cc04bd1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:48:47.956582 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-4fc60ac2-dd63-4948-87a2-7066794973f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:47.957216 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-4fc60ac2-dd63-4948-87a2-7066794973f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:47.958735 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1071/2134] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:48:47 2022] DELETE /identity/v3/projects/dab2b35616fc41ff8aeb4b534cc04bd1 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 10:48:47.963444 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5110630-234d-4b16-85d3-7c8368763c87 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:47.963731 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5110630-234d-4b16-85d3-7c8368763c87 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:47.964004 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5110630-234d-4b16-85d3-7c8368763c87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:47.965191 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-89dadc62-1963-4695-98a0-33fff20b062b None None] MFA Rules not processed for user `84b8e4ab226747cb9bba2a31a54c0d81`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:48:47.966407 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-89dadc62-1963-4695-98a0-33fff20b062b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:47.982978 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-a5110630-234d-4b16-85d3-7c8368763c87 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:48:47.984757 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-a5110630-234d-4b16-85d3-7c8368763c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:48.035807 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:48:48.039076 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1064/2135] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:48:47 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 10:48:48.051090 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dadc62-1963-4695-98a0-33fff20b062b None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:48.051890 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1072/2136] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:48:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 10:48:48.052823 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-89dadc62-1963-4695-98a0-33fff20b062b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:48.059981 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9892321e-5478-445b-9566-35ee9cbf5c9c None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:48.060318 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9892321e-5478-445b-9566-35ee9cbf5c9c None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:48.060608 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9892321e-5478-445b-9566-35ee9cbf5c9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:48.081213 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-9892321e-5478-445b-9566-35ee9cbf5c9c None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:48:48.082475 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-9892321e-5478-445b-9566-35ee9cbf5c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:48.090949 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1073/2137] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:48:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:48:48.095731 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9892321e-5478-445b-9566-35ee9cbf5c9c None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:48.105483 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9892321e-5478-445b-9566-35ee9cbf5c9c None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:48.110698 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99916a2c-8a30-4fe8-9915-57acd7391496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:48.111516 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99916a2c-8a30-4fe8-9915-57acd7391496 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:48.111986 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99916a2c-8a30-4fe8-9915-57acd7391496 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:48.112298 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99916a2c-8a30-4fe8-9915-57acd7391496 admin admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:48.112949 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99916a2c-8a30-4fe8-9915-57acd7391496 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:48:48.113255 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99916a2c-8a30-4fe8-9915-57acd7391496 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:48.113592 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dadc62-1963-4695-98a0-33fff20b062b None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:48.114970 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99916a2c-8a30-4fe8-9915-57acd7391496 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:48.117282 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e58326c-4e39-49c6-be6a-56e0f97df291 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:48.117879 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e58326c-4e39-49c6-be6a-56e0f97df291 service cinder] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:48.118155 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e58326c-4e39-49c6-be6a-56e0f97df291 service cinder] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:48.118423 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e58326c-4e39-49c6-be6a-56e0f97df291 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:48.128748 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1074/2138] 10.222.0.15 () {58 vars in 1153 bytes} [Sun Sep 11 10:48:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:48:48.128896 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e58326c-4e39-49c6-be6a-56e0f97df291 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:48.130151 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e58326c-4e39-49c6-be6a-56e0f97df291 service cinder] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:48.133594 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99916a2c-8a30-4fe8-9915-57acd7391496 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:48.141235 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1065/2139] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:48:48 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:48:48.146674 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99916a2c-8a30-4fe8-9915-57acd7391496 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:48.151296 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fb1ba8-ce11-464e-985f-440959ee0676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:48.151929 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb1ba8-ce11-464e-985f-440959ee0676 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:48.152217 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb1ba8-ce11-464e-985f-440959ee0676 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:48.152492 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb1ba8-ce11-464e-985f-440959ee0676 admin admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:48.153123 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fb1ba8-ce11-464e-985f-440959ee0676 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:48.155015 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fb1ba8-ce11-464e-985f-440959ee0676 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:48.187553 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1075/2140] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:48:48 2022] POST /identity/v3/projects => generated 371 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 10:48:48.192731 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e58326c-4e39-49c6-be6a-56e0f97df291 service cinder] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:48.204073 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e58326c-4e39-49c6-be6a-56e0f97df291 service cinder] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:48.226094 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fb1ba8-ce11-464e-985f-440959ee0676 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:48.269804 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943de90e-ed13-444c-ad92-9c0cfad50418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:48.270575 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943de90e-ed13-444c-ad92-9c0cfad50418 admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:48.270859 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943de90e-ed13-444c-ad92-9c0cfad50418 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:48.271353 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943de90e-ed13-444c-ad92-9c0cfad50418 admin admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:48.272005 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943de90e-ed13-444c-ad92-9c0cfad50418 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:48.273974 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943de90e-ed13-444c-ad92-9c0cfad50418 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:48.282389 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fb1ba8-ce11-464e-985f-440959ee0676 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:48.283708 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:48:48.287235 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a5db64-a0b2-4473-98aa-80a656908a4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:48.287840 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a5db64-a0b2-4473-98aa-80a656908a4f service cinder] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:48.288124 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a5db64-a0b2-4473-98aa-80a656908a4f service cinder] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:48.288446 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a5db64-a0b2-4473-98aa-80a656908a4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:48.290388 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-90a5db64-a0b2-4473-98aa-80a656908a4f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:48.298414 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1066/2141] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:48:48 2022] POST /identity/v3/users => generated 286 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:48:48.302400 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943de90e-ed13-444c-ad92-9c0cfad50418 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:48.313001 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943de90e-ed13-444c-ad92-9c0cfad50418 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:48.318086 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a3bd75-fd00-486e-b868-74c543fcfc75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:48.318891 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a3bd75-fd00-486e-b868-74c543fcfc75 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:48.319302 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a3bd75-fd00-486e-b868-74c543fcfc75 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:48.319699 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a3bd75-fd00-486e-b868-74c543fcfc75 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:48.320585 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a3bd75-fd00-486e-b868-74c543fcfc75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:48.322678 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a3bd75-fd00-486e-b868-74c543fcfc75 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:48.337748 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1067/2142] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:48:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:48:48.338701 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a3bd75-fd00-486e-b868-74c543fcfc75 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:48.348754 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a3bd75-fd00-486e-b868-74c543fcfc75 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:48.352987 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a5db64-a0b2-4473-98aa-80a656908a4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:48.353356 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08af3632-2c36-4304-a2f2-b76aee6a25b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:48.353939 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08af3632-2c36-4304-a2f2-b76aee6a25b1 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:48.355731 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08af3632-2c36-4304-a2f2-b76aee6a25b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:48.355920 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a5db64-a0b2-4473-98aa-80a656908a4f service cinder] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:48.356006 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08af3632-2c36-4304-a2f2-b76aee6a25b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:48.356006 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08af3632-2c36-4304-a2f2-b76aee6a25b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:48.356725 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08af3632-2c36-4304-a2f2-b76aee6a25b1 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:48.368532 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1068/2143] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:48:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 10:48:48.371564 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1076/2144] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:48:48 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 10:48:48.372774 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08af3632-2c36-4304-a2f2-b76aee6a25b1 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:48.384780 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08af3632-2c36-4304-a2f2-b76aee6a25b1 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:48.389789 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f14d7b-65fe-4929-ba45-97d8aae884e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:48.390485 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f14d7b-65fe-4929-ba45-97d8aae884e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:48.390768 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f14d7b-65fe-4929-ba45-97d8aae884e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:48.391046 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f14d7b-65fe-4929-ba45-97d8aae884e7 admin admin] PATH_INFO: `/v3/projects/6e5993a0a4964d7597bf4e2e8f252171/users/30f5e5a2ce8749d692600f1d2ab4fdb9/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:48.408808 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f14d7b-65fe-4929-ba45-97d8aae884e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e5993a0a4964d7597bf4e2e8f252171, user_id=30f5e5a2ce8749d692600f1d2ab4fdb9, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:48.411781 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f14d7b-65fe-4929-ba45-97d8aae884e7 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:48.426843 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1069/2145] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:48:48 2022] PUT /identity/v3/projects/6e5993a0a4964d7597bf4e2e8f252171/users/30f5e5a2ce8749d692600f1d2ab4fdb9/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:48:48.432094 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dac7fa-6455-40fb-9f6b-1e57710e9466 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:48.432403 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dac7fa-6455-40fb-9f6b-1e57710e9466 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:48.432743 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dac7fa-6455-40fb-9f6b-1e57710e9466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:48.456370 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-d9dac7fa-6455-40fb-9f6b-1e57710e9466 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:48:48.457844 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-d9dac7fa-6455-40fb-9f6b-1e57710e9466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:48.543359 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1077/2146] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:48:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:48:48.552160 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f14d7b-65fe-4929-ba45-97d8aae884e7 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:48.553934 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-22f14d7b-65fe-4929-ba45-97d8aae884e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:48.617728 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f14d7b-65fe-4929-ba45-97d8aae884e7 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:48.621349 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca2a8f2-9423-42d0-9a51-509f69136cf6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:48.621948 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca2a8f2-9423-42d0-9a51-509f69136cf6 service neutron] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:48.622222 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca2a8f2-9423-42d0-9a51-509f69136cf6 service neutron] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:48.622491 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca2a8f2-9423-42d0-9a51-509f69136cf6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:48.631783 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca2a8f2-9423-42d0-9a51-509f69136cf6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:48.633189 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca2a8f2-9423-42d0-9a51-509f69136cf6 service neutron] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:48.643176 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1070/2147] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:48:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:48:48.884528 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dac7fa-6455-40fb-9f6b-1e57710e9466 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:48.885995 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-d9dac7fa-6455-40fb-9f6b-1e57710e9466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:48.900287 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dac7fa-6455-40fb-9f6b-1e57710e9466 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:48.907934 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b52d208-5cc8-4552-a7e1-19ccbabe7096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:48.908684 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b52d208-5cc8-4552-a7e1-19ccbabe7096 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:48.909043 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b52d208-5cc8-4552-a7e1-19ccbabe7096 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:48.909393 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b52d208-5cc8-4552-a7e1-19ccbabe7096 admin admin] PATH_INFO: `/v3/users/ef7423bd14644dcca25142af6392ac1c` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:48.913372 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b52d208-5cc8-4552-a7e1-19ccbabe7096 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef7423bd14644dcca25142af6392ac1c)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:48.915377 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b52d208-5cc8-4552-a7e1-19ccbabe7096 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:48.922716 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:48:49.000114 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-4b52d208-5cc8-4552-a7e1-19ccbabe7096 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef7423bd14644dcca25142af6392ac1c'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:49.016029 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-4b52d208-5cc8-4552-a7e1-19ccbabe7096 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef7423bd14644dcca25142af6392ac1c'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:49.016901 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-4b52d208-5cc8-4552-a7e1-19ccbabe7096 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef7423bd14644dcca25142af6392ac1c'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:49.018132 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-4b52d208-5cc8-4552-a7e1-19ccbabe7096 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef7423bd14644dcca25142af6392ac1c'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:49.032775 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-4b52d208-5cc8-4552-a7e1-19ccbabe7096 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef7423bd14644dcca25142af6392ac1c'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:49.047548 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1078/2148] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:48:48 2022] DELETE /identity/v3/users/ef7423bd14644dcca25142af6392ac1c => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:48:49.052177 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5588cac6-cd6b-4226-adca-a35964b5f7ce None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:49.052625 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5588cac6-cd6b-4226-adca-a35964b5f7ce None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:49.052625 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5588cac6-cd6b-4226-adca-a35964b5f7ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:49.073595 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-5588cac6-cd6b-4226-adca-a35964b5f7ce None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:48:49.074831 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-5588cac6-cd6b-4226-adca-a35964b5f7ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:49.144703 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1071/2149] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:48:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:48:49.154611 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b52d208-5cc8-4552-a7e1-19ccbabe7096 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:49.156494 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-4b52d208-5cc8-4552-a7e1-19ccbabe7096 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:49.217179 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b52d208-5cc8-4552-a7e1-19ccbabe7096 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:49.221354 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb10ffbc-cfe5-4fcd-b28e-376d07f42ad4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:49.221764 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb10ffbc-cfe5-4fcd-b28e-376d07f42ad4 service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:49.222045 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb10ffbc-cfe5-4fcd-b28e-376d07f42ad4 service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:49.222318 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb10ffbc-cfe5-4fcd-b28e-376d07f42ad4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:49.232659 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb10ffbc-cfe5-4fcd-b28e-376d07f42ad4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:49.234161 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb10ffbc-cfe5-4fcd-b28e-376d07f42ad4 service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:49.244379 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1079/2150] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:48:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:48:49.551725 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5588cac6-cd6b-4226-adca-a35964b5f7ce None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:49.552021 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-5588cac6-cd6b-4226-adca-a35964b5f7ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:49.591670 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5588cac6-cd6b-4226-adca-a35964b5f7ce None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:49.598188 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b8e844-fe3e-4608-8d09-86d39ebac8f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:49.598927 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b8e844-fe3e-4608-8d09-86d39ebac8f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:49.599943 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b8e844-fe3e-4608-8d09-86d39ebac8f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:49.600496 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b8e844-fe3e-4608-8d09-86d39ebac8f5 admin admin] PATH_INFO: `/v3/projects/072a17fe2391408996ec02c5723f04ee` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:49.604687 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b8e844-fe3e-4608-8d09-86d39ebac8f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=072a17fe2391408996ec02c5723f04ee)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:49.606667 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d74886-4abb-4ff9-affd-835eb9d9fcef None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:49.606969 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d74886-4abb-4ff9-affd-835eb9d9fcef None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:49.607547 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d74886-4abb-4ff9-affd-835eb9d9fcef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:49.609995 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b8e844-fe3e-4608-8d09-86d39ebac8f5 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:49.649635 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-c1d74886-4abb-4ff9-affd-835eb9d9fcef None None] MFA Rules not processed for user `30f5e5a2ce8749d692600f1d2ab4fdb9`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:48:49.651353 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-c1d74886-4abb-4ff9-affd-835eb9d9fcef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:49.681696 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:INSERT=4,keystone:SELECT=93,keystone:DELETE=3 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:48:49.689416 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-b8b8e844-fe3e-4608-8d09-86d39ebac8f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '072a17fe2391408996ec02c5723f04ee', 'request_id': 'req-b8b8e844-fe3e-4608-8d09-86d39ebac8f5'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:49.704528 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-b8b8e844-fe3e-4608-8d09-86d39ebac8f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '072a17fe2391408996ec02c5723f04ee', 'request_id': 'req-b8b8e844-fe3e-4608-8d09-86d39ebac8f5'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:49.716738 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-b8b8e844-fe3e-4608-8d09-86d39ebac8f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '072a17fe2391408996ec02c5723f04ee', 'request_id': 'req-b8b8e844-fe3e-4608-8d09-86d39ebac8f5'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:49.738906 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-b8b8e844-fe3e-4608-8d09-86d39ebac8f5 admin admin] The token cache is being invalidate because project 072a17fe2391408996ec02c5723f04ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90653) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:48:49.739351 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-b8b8e844-fe3e-4608-8d09-86d39ebac8f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:49.740443 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-b8b8e844-fe3e-4608-8d09-86d39ebac8f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:49.743164 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1072/2151] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:48:49 2022] DELETE /identity/v3/projects/072a17fe2391408996ec02c5723f04ee => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:48:49.747731 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33147a3-6d46-40f6-98fa-e781b2db19f7 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:49.747984 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33147a3-6d46-40f6-98fa-e781b2db19f7 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:49.748264 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33147a3-6d46-40f6-98fa-e781b2db19f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:49.791660 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-b33147a3-6d46-40f6-98fa-e781b2db19f7 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:48:49.793113 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-b33147a3-6d46-40f6-98fa-e781b2db19f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:49.799270 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1080/2152] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:48:49 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 10:48:49.811549 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d74886-4abb-4ff9-affd-835eb9d9fcef None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:49.815006 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-c1d74886-4abb-4ff9-affd-835eb9d9fcef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:49.840123 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:48:49.884055 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1073/2153] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:48:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 10:48:49.886883 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3376d8c-3dc5-441e-af48-694aa2dc7d48 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:49.887297 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3376d8c-3dc5-441e-af48-694aa2dc7d48 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:49.887645 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3376d8c-3dc5-441e-af48-694aa2dc7d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:49.910023 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d74886-4abb-4ff9-affd-835eb9d9fcef None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:49.911824 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-d3376d8c-3dc5-441e-af48-694aa2dc7d48 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:48:49.913070 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-d3376d8c-3dc5-441e-af48-694aa2dc7d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:49.922966 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1074/2154] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:48:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:48:49.926936 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3376d8c-3dc5-441e-af48-694aa2dc7d48 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:49.936447 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3376d8c-3dc5-441e-af48-694aa2dc7d48 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:49.942755 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d298d45-e6a1-42ea-935b-ad180d8acfd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:49.943508 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d298d45-e6a1-42ea-935b-ad180d8acfd4 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:49.943850 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d298d45-e6a1-42ea-935b-ad180d8acfd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:49.944307 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d298d45-e6a1-42ea-935b-ad180d8acfd4 admin admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:49.945071 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d298d45-e6a1-42ea-935b-ad180d8acfd4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:48:49.945477 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d298d45-e6a1-42ea-935b-ad180d8acfd4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:49.947404 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d298d45-e6a1-42ea-935b-ad180d8acfd4 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:49.961791 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1075/2155] 10.222.0.15 () {58 vars in 1153 bytes} [Sun Sep 11 10:48:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:48:49.965238 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd45794-46bc-464d-9b4f-73d1cf3bd18f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:49.965946 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d298d45-e6a1-42ea-935b-ad180d8acfd4 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:49.966093 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd45794-46bc-464d-9b4f-73d1cf3bd18f service cinder] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:49.966271 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd45794-46bc-464d-9b4f-73d1cf3bd18f service cinder] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:49.966538 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd45794-46bc-464d-9b4f-73d1cf3bd18f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:49.979352 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d298d45-e6a1-42ea-935b-ad180d8acfd4 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:49.986950 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62b257e-4b8c-43bc-be5f-ad46bcc83653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:49.987825 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62b257e-4b8c-43bc-be5f-ad46bcc83653 admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:49.988284 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62b257e-4b8c-43bc-be5f-ad46bcc83653 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:49.988765 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62b257e-4b8c-43bc-be5f-ad46bcc83653 admin admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:49.989495 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62b257e-4b8c-43bc-be5f-ad46bcc83653 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:49.992508 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62b257e-4b8c-43bc-be5f-ad46bcc83653 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:50.058484 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd45794-46bc-464d-9b4f-73d1cf3bd18f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:50.060099 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd45794-46bc-464d-9b4f-73d1cf3bd18f service cinder] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:50.063517 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1076/2156] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:48:49 2022] POST /identity/v3/projects => generated 373 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 10:48:50.067543 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62b257e-4b8c-43bc-be5f-ad46bcc83653 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:50.073926 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1081/2157] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:48:49 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 10:48:50.168135 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62b257e-4b8c-43bc-be5f-ad46bcc83653 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:50.175575 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9121df13-537e-43c9-9c63-08b4bd9d6423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:50.177028 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9121df13-537e-43c9-9c63-08b4bd9d6423 admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:50.177313 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9121df13-537e-43c9-9c63-08b4bd9d6423 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:50.177586 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9121df13-537e-43c9-9c63-08b4bd9d6423 admin admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:50.178350 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9121df13-537e-43c9-9c63-08b4bd9d6423 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:50.181600 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9121df13-537e-43c9-9c63-08b4bd9d6423 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:50.209787 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1077/2158] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:48:50 2022] POST /identity/v3/users => generated 287 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:48:50.211021 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd45794-46bc-464d-9b4f-73d1cf3bd18f service cinder] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:50.223223 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd45794-46bc-464d-9b4f-73d1cf3bd18f service cinder] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:50.228022 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0727f5-ef98-45e1-a28e-93ba309d0b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:50.228767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0727f5-ef98-45e1-a28e-93ba309d0b42 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:50.229050 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0727f5-ef98-45e1-a28e-93ba309d0b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:50.229434 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0727f5-ef98-45e1-a28e-93ba309d0b42 admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:50.230063 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0727f5-ef98-45e1-a28e-93ba309d0b42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:50.231661 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0727f5-ef98-45e1-a28e-93ba309d0b42 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:50.243466 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1082/2159] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:48:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:48:50.247002 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9121df13-537e-43c9-9c63-08b4bd9d6423 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:50.257703 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9121df13-537e-43c9-9c63-08b4bd9d6423 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:50.266849 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd32b3b-d6e0-45c4-9e57-dc73fbd1d56f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:50.267480 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd32b3b-d6e0-45c4-9e57-dc73fbd1d56f admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:50.267771 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd32b3b-d6e0-45c4-9e57-dc73fbd1d56f admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:50.268048 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd32b3b-d6e0-45c4-9e57-dc73fbd1d56f admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:50.268713 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd32b3b-d6e0-45c4-9e57-dc73fbd1d56f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:50.270657 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd32b3b-d6e0-45c4-9e57-dc73fbd1d56f admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:50.283472 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1078/2160] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:48:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 10:48:50.287126 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0727f5-ef98-45e1-a28e-93ba309d0b42 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:50.300097 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0727f5-ef98-45e1-a28e-93ba309d0b42 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:50.304573 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e37d024-8104-4bbe-a06f-0248c0841430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:50.305237 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e37d024-8104-4bbe-a06f-0248c0841430 admin admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:50.305523 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e37d024-8104-4bbe-a06f-0248c0841430 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:50.305805 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e37d024-8104-4bbe-a06f-0248c0841430 admin admin] PATH_INFO: `/v3/projects/d166a371f57a462aad46580e6f81e474/users/f9c5cf874559479f87137164ac3cb75d/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:50.322253 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e37d024-8104-4bbe-a06f-0248c0841430 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d166a371f57a462aad46580e6f81e474, user_id=f9c5cf874559479f87137164ac3cb75d, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:50.325326 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e37d024-8104-4bbe-a06f-0248c0841430 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:50.342452 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1083/2161] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:48:50 2022] PUT /identity/v3/projects/d166a371f57a462aad46580e6f81e474/users/f9c5cf874559479f87137164ac3cb75d/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:48:50.356463 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c37a7d7-6835-49c2-b26b-273c7fae2c74 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:50.356848 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c37a7d7-6835-49c2-b26b-273c7fae2c74 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:50.357425 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c37a7d7-6835-49c2-b26b-273c7fae2c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:50.372737 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:48:50.393579 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-6c37a7d7-6835-49c2-b26b-273c7fae2c74 None None] MFA Rules not processed for user `f9c5cf874559479f87137164ac3cb75d`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:48:50.394927 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-6c37a7d7-6835-49c2-b26b-273c7fae2c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:50.473801 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1079/2162] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:48:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:48:50.487632 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e37d024-8104-4bbe-a06f-0248c0841430 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:50.542159 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e37d024-8104-4bbe-a06f-0248c0841430 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:50.545979 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d0314c-df43-4ac6-96b8-68daa0a6f394 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:50.547117 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0314c-df43-4ac6-96b8-68daa0a6f394 service cinder] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:50.547391 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0314c-df43-4ac6-96b8-68daa0a6f394 service cinder] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:50.547692 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0314c-df43-4ac6-96b8-68daa0a6f394 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:50.552851 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:48:50.557269 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d0314c-df43-4ac6-96b8-68daa0a6f394 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:50.559369 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d0314c-df43-4ac6-96b8-68daa0a6f394 service cinder] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:50.568726 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1084/2163] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:48:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:48:50.594053 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c37a7d7-6835-49c2-b26b-273c7fae2c74 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:50.656885 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c37a7d7-6835-49c2-b26b-273c7fae2c74 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:50.661534 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6d8580-6f6d-4e7c-b412-520475b69b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:50.662175 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6d8580-6f6d-4e7c-b412-520475b69b8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:50.662440 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6d8580-6f6d-4e7c-b412-520475b69b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:50.662712 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6d8580-6f6d-4e7c-b412-520475b69b8a admin admin] PATH_INFO: `/v3/users/f9c5cf874559479f87137164ac3cb75d` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:50.663870 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6d8580-6f6d-4e7c-b412-520475b69b8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9c5cf874559479f87137164ac3cb75d)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:50.665790 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6d8580-6f6d-4e7c-b412-520475b69b8a admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:50.711978 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-0d6d8580-6f6d-4e7c-b412-520475b69b8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9c5cf874559479f87137164ac3cb75d'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:50.720336 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-0d6d8580-6f6d-4e7c-b412-520475b69b8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9c5cf874559479f87137164ac3cb75d'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:50.720986 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-0d6d8580-6f6d-4e7c-b412-520475b69b8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9c5cf874559479f87137164ac3cb75d'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:50.721620 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-0d6d8580-6f6d-4e7c-b412-520475b69b8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9c5cf874559479f87137164ac3cb75d'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:50.728647 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-0d6d8580-6f6d-4e7c-b412-520475b69b8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9c5cf874559479f87137164ac3cb75d'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:50.746462 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1080/2164] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:48:50 2022] DELETE /identity/v3/users/f9c5cf874559479f87137164ac3cb75d => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:48:50.751171 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab63afc-01b7-4bb5-ab26-33c6d6fc359e None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:50.751484 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab63afc-01b7-4bb5-ab26-33c6d6fc359e None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:50.751809 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab63afc-01b7-4bb5-ab26-33c6d6fc359e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:50.770789 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-eab63afc-01b7-4bb5-ab26-33c6d6fc359e None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:48:50.772017 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-eab63afc-01b7-4bb5-ab26-33c6d6fc359e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:50.830105 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1085/2165] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:48:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:48:50.839910 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6d8580-6f6d-4e7c-b412-520475b69b8a admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:50.841751 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-0d6d8580-6f6d-4e7c-b412-520475b69b8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:50.862670 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:48:50.904680 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6d8580-6f6d-4e7c-b412-520475b69b8a admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:50.909196 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841de8fc-febd-4689-bf3c-d3d0578dca66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:50.909802 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841de8fc-febd-4689-bf3c-d3d0578dca66 service neutron] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:50.910081 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841de8fc-febd-4689-bf3c-d3d0578dca66 service neutron] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:50.910352 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841de8fc-febd-4689-bf3c-d3d0578dca66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:50.919675 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841de8fc-febd-4689-bf3c-d3d0578dca66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:50.921286 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841de8fc-febd-4689-bf3c-d3d0578dca66 service neutron] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:50.931090 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1081/2166] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:48:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:48:51.284361 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab63afc-01b7-4bb5-ab26-33c6d6fc359e None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:51.286340 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-eab63afc-01b7-4bb5-ab26-33c6d6fc359e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:51.300342 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab63afc-01b7-4bb5-ab26-33c6d6fc359e None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:51.305726 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff4582e-013e-4021-8879-f3221fcaf200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:51.312652 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff4582e-013e-4021-8879-f3221fcaf200 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:51.313026 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff4582e-013e-4021-8879-f3221fcaf200 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:51.313373 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff4582e-013e-4021-8879-f3221fcaf200 admin admin] PATH_INFO: `/v3/projects/d166a371f57a462aad46580e6f81e474` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:51.314638 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff4582e-013e-4021-8879-f3221fcaf200 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d166a371f57a462aad46580e6f81e474)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:51.316617 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff4582e-013e-4021-8879-f3221fcaf200 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:51.368118 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-8ff4582e-013e-4021-8879-f3221fcaf200 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd166a371f57a462aad46580e6f81e474', 'request_id': 'req-8ff4582e-013e-4021-8879-f3221fcaf200'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:51.375284 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-8ff4582e-013e-4021-8879-f3221fcaf200 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd166a371f57a462aad46580e6f81e474', 'request_id': 'req-8ff4582e-013e-4021-8879-f3221fcaf200'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:51.380756 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-8ff4582e-013e-4021-8879-f3221fcaf200 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd166a371f57a462aad46580e6f81e474', 'request_id': 'req-8ff4582e-013e-4021-8879-f3221fcaf200'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:51.394111 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-8ff4582e-013e-4021-8879-f3221fcaf200 admin admin] The token cache is being invalidate because project d166a371f57a462aad46580e6f81e474 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:48:51.394334 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-8ff4582e-013e-4021-8879-f3221fcaf200 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:51.394955 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-8ff4582e-013e-4021-8879-f3221fcaf200 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:48:51.396498 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1086/2167] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:48:51 2022] DELETE /identity/v3/projects/d166a371f57a462aad46580e6f81e474 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:48:51.401322 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be824d2-0cda-427c-bd15-d0274821fda8 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:51.401804 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be824d2-0cda-427c-bd15-d0274821fda8 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:51.401905 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be824d2-0cda-427c-bd15-d0274821fda8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:51.421253 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-0be824d2-0cda-427c-bd15-d0274821fda8 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:48:51.422557 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-0be824d2-0cda-427c-bd15-d0274821fda8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:51.485167 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1082/2168] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:48:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:48:51.492797 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d8beee-de8b-463f-bfce-386ca680cbd6 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:51.493151 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d8beee-de8b-463f-bfce-386ca680cbd6 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:51.493519 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d8beee-de8b-463f-bfce-386ca680cbd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:51.514256 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-03d8beee-de8b-463f-bfce-386ca680cbd6 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:48:51.516366 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-03d8beee-de8b-463f-bfce-386ca680cbd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:51.524324 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1087/2169] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:48:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:48:51.528805 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be824d2-0cda-427c-bd15-d0274821fda8 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:51.538641 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be824d2-0cda-427c-bd15-d0274821fda8 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:51.543280 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16449d2-ae57-41c7-b9b8-23764227dbea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:51.543913 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16449d2-ae57-41c7-b9b8-23764227dbea admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:51.544257 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16449d2-ae57-41c7-b9b8-23764227dbea admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:51.544525 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16449d2-ae57-41c7-b9b8-23764227dbea admin admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:51.545210 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16449d2-ae57-41c7-b9b8-23764227dbea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:48:51.545566 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16449d2-ae57-41c7-b9b8-23764227dbea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:51.547544 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16449d2-ae57-41c7-b9b8-23764227dbea admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:51.559243 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1083/2170] 10.222.0.15 () {58 vars in 1153 bytes} [Sun Sep 11 10:48:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:48:51.564048 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d8beee-de8b-463f-bfce-386ca680cbd6 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:51.573231 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d8beee-de8b-463f-bfce-386ca680cbd6 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:51.578670 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3082b9-4289-4921-b4d1-b1eb764b79b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:51.579266 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3082b9-4289-4921-b4d1-b1eb764b79b7 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:51.579582 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3082b9-4289-4921-b4d1-b1eb764b79b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:51.579995 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3082b9-4289-4921-b4d1-b1eb764b79b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:51.580612 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3082b9-4289-4921-b4d1-b1eb764b79b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:51.582576 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3082b9-4289-4921-b4d1-b1eb764b79b7 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:51.610928 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1088/2171] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:48:51 2022] POST /identity/v3/projects => generated 399 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:48:51.614904 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16449d2-ae57-41c7-b9b8-23764227dbea admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:51.625259 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16449d2-ae57-41c7-b9b8-23764227dbea admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:51.682596 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9594950b-a4b0-413e-8439-37af214217dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:51.683113 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9594950b-a4b0-413e-8439-37af214217dc admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:51.683393 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9594950b-a4b0-413e-8439-37af214217dc admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:51.683691 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9594950b-a4b0-413e-8439-37af214217dc admin admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:51.684308 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9594950b-a4b0-413e-8439-37af214217dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:51.686206 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9594950b-a4b0-413e-8439-37af214217dc admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:51.706986 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1084/2172] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:48:51 2022] POST /identity/v3/users => generated 300 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 10:48:51.711125 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3082b9-4289-4921-b4d1-b1eb764b79b7 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:51.721419 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3082b9-4289-4921-b4d1-b1eb764b79b7 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:51.725967 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519d3009-0e76-4a97-b71f-cd32580f6b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:51.726592 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519d3009-0e76-4a97-b71f-cd32580f6b46 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:51.726872 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519d3009-0e76-4a97-b71f-cd32580f6b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:51.727144 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519d3009-0e76-4a97-b71f-cd32580f6b46 admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:51.727808 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519d3009-0e76-4a97-b71f-cd32580f6b46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:51.729404 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519d3009-0e76-4a97-b71f-cd32580f6b46 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:51.740371 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1089/2173] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:48:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:48:51.744127 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9594950b-a4b0-413e-8439-37af214217dc admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:51.756449 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9594950b-a4b0-413e-8439-37af214217dc admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:51.761036 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c61f9c-1703-4955-9f42-4191bfbfbdf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:51.761628 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c61f9c-1703-4955-9f42-4191bfbfbdf6 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:51.761905 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c61f9c-1703-4955-9f42-4191bfbfbdf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:51.762171 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c61f9c-1703-4955-9f42-4191bfbfbdf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:51.762943 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c61f9c-1703-4955-9f42-4191bfbfbdf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:51.764484 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c61f9c-1703-4955-9f42-4191bfbfbdf6 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:51.776483 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1085/2174] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:48:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:48:51.780818 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519d3009-0e76-4a97-b71f-cd32580f6b46 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:51.791237 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519d3009-0e76-4a97-b71f-cd32580f6b46 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:51.796615 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73725aff-678e-4541-b1e6-c1570314fefd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:51.798069 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73725aff-678e-4541-b1e6-c1570314fefd admin admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:51.798069 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73725aff-678e-4541-b1e6-c1570314fefd admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:51.798284 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73725aff-678e-4541-b1e6-c1570314fefd admin admin] PATH_INFO: `/v3/projects/347919e6f5604c62b60e7d5e05a30a20/users/4e0fb6afb8f940cc959a9bc90c6eaa62/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:51.814770 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73725aff-678e-4541-b1e6-c1570314fefd admin admin] RBAC: Authorizing `identity:create_grant(project_id=347919e6f5604c62b60e7d5e05a30a20, user_id=4e0fb6afb8f940cc959a9bc90c6eaa62, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:51.818278 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73725aff-678e-4541-b1e6-c1570314fefd admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:51.832310 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1090/2175] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:48:51 2022] PUT /identity/v3/projects/347919e6f5604c62b60e7d5e05a30a20/users/4e0fb6afb8f940cc959a9bc90c6eaa62/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:48:51.842667 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57182cdc-918a-4a3f-a4e6-74161f7d7c9d None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:51.843171 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57182cdc-918a-4a3f-a4e6-74161f7d7c9d None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:51.843535 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57182cdc-918a-4a3f-a4e6-74161f7d7c9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:51.871497 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-57182cdc-918a-4a3f-a4e6-74161f7d7c9d None None] MFA Rules not processed for user `4e0fb6afb8f940cc959a9bc90c6eaa62`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:48:51.872735 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-57182cdc-918a-4a3f-a4e6-74161f7d7c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:51.885815 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:48:51.941385 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1086/2176] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:48:51 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:48:51.953387 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73725aff-678e-4541-b1e6-c1570314fefd admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:48:51.955096 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-73725aff-678e-4541-b1e6-c1570314fefd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:51.969900 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:48:52.013386 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73725aff-678e-4541-b1e6-c1570314fefd admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:48:52.017723 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d561481-2f6c-4f9c-9bb7-e6799763bcfe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:48:52.018328 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d561481-2f6c-4f9c-9bb7-e6799763bcfe service cinder] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:52.018610 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d561481-2f6c-4f9c-9bb7-e6799763bcfe service cinder] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:52.018885 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d561481-2f6c-4f9c-9bb7-e6799763bcfe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:52.027090 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d561481-2f6c-4f9c-9bb7-e6799763bcfe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:48:52.028550 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d561481-2f6c-4f9c-9bb7-e6799763bcfe service cinder] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:48:52.037985 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1091/2177] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:48:51 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:48:52.739028 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235cbca4-b221-4f33-b3b5-a7fda0e21625 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:48:52.739698 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235cbca4-b221-4f33-b3b5-a7fda0e21625 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:48:52.739698 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235cbca4-b221-4f33-b3b5-a7fda0e21625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:48:52.759323 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-235cbca4-b221-4f33-b3b5-a7fda0e21625 None None] MFA Rules not processed for user `30f5e5a2ce8749d692600f1d2ab4fdb9`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:48:52.760597 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-235cbca4-b221-4f33-b3b5-a7fda0e21625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:48:52.824368 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1087/2178] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:48:52 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:49:02.033946 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=20 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:49:02.814819 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=46 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:49:23.342773 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d561481-2f6c-4f9c-9bb7-e6799763bcfe service cinder] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:49:23.404891 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d561481-2f6c-4f9c-9bb7-e6799763bcfe service cinder] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:49:23.409517 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36d8906-7b80-4cc7-9765-72d6f2f49e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:49:23.410157 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36d8906-7b80-4cc7-9765-72d6f2f49e3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:49:23.410436 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36d8906-7b80-4cc7-9765-72d6f2f49e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:49:23.410709 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36d8906-7b80-4cc7-9765-72d6f2f49e3f admin admin] PATH_INFO: `/v3/users/4e0fb6afb8f940cc959a9bc90c6eaa62` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:49:23.411857 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36d8906-7b80-4cc7-9765-72d6f2f49e3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e0fb6afb8f940cc959a9bc90c6eaa62)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:49:23.413769 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36d8906-7b80-4cc7-9765-72d6f2f49e3f admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:49:23.461783 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-c36d8906-7b80-4cc7-9765-72d6f2f49e3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e0fb6afb8f940cc959a9bc90c6eaa62'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:49:23.470013 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-c36d8906-7b80-4cc7-9765-72d6f2f49e3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e0fb6afb8f940cc959a9bc90c6eaa62'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:49:23.470675 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-c36d8906-7b80-4cc7-9765-72d6f2f49e3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e0fb6afb8f940cc959a9bc90c6eaa62'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:49:23.471288 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-c36d8906-7b80-4cc7-9765-72d6f2f49e3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e0fb6afb8f940cc959a9bc90c6eaa62'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:49:23.489089 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-c36d8906-7b80-4cc7-9765-72d6f2f49e3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e0fb6afb8f940cc959a9bc90c6eaa62'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:49:23.503315 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1092/2179] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:49:23 2022] DELETE /identity/v3/users/4e0fb6afb8f940cc959a9bc90c6eaa62 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:49:23.508160 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810fc50d-57e7-4878-8b70-c3c169054525 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:49:23.508643 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810fc50d-57e7-4878-8b70-c3c169054525 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:49:23.508739 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810fc50d-57e7-4878-8b70-c3c169054525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:49:23.533238 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-810fc50d-57e7-4878-8b70-c3c169054525 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:49:23.534527 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-810fc50d-57e7-4878-8b70-c3c169054525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:49:23.651607 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1088/2180] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:49:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:49:23.658533 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36d8906-7b80-4cc7-9765-72d6f2f49e3f admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:49:23.660388 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-c36d8906-7b80-4cc7-9765-72d6f2f49e3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:49:23.757976 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36d8906-7b80-4cc7-9765-72d6f2f49e3f admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:49:23.767759 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0c8765-b293-4684-b541-a30c76fc1c07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:49:23.768480 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0c8765-b293-4684-b541-a30c76fc1c07 service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:49:23.769041 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0c8765-b293-4684-b541-a30c76fc1c07 service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:49:23.769233 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0c8765-b293-4684-b541-a30c76fc1c07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:49:23.783807 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0c8765-b293-4684-b541-a30c76fc1c07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:49:23.785550 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0c8765-b293-4684-b541-a30c76fc1c07 service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:49:23.799585 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1093/2181] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:49:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:49:24.104688 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810fc50d-57e7-4878-8b70-c3c169054525 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:49:24.110947 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-810fc50d-57e7-4878-8b70-c3c169054525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:49:24.142934 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810fc50d-57e7-4878-8b70-c3c169054525 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:49:24.148475 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fee5d1-e372-432e-9d3e-a5edd741beff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:49:24.149124 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fee5d1-e372-432e-9d3e-a5edd741beff admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:49:24.149402 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fee5d1-e372-432e-9d3e-a5edd741beff admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:49:24.149672 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fee5d1-e372-432e-9d3e-a5edd741beff admin admin] PATH_INFO: `/v3/projects/347919e6f5604c62b60e7d5e05a30a20` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:49:24.150818 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fee5d1-e372-432e-9d3e-a5edd741beff admin admin] RBAC: Authorizing `identity:delete_project(project_id=347919e6f5604c62b60e7d5e05a30a20)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:49:24.163650 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fee5d1-e372-432e-9d3e-a5edd741beff admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:49:24.206680 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-c5fee5d1-e372-432e-9d3e-a5edd741beff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '347919e6f5604c62b60e7d5e05a30a20', 'request_id': 'req-c5fee5d1-e372-432e-9d3e-a5edd741beff'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:49:24.214730 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-c5fee5d1-e372-432e-9d3e-a5edd741beff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '347919e6f5604c62b60e7d5e05a30a20', 'request_id': 'req-c5fee5d1-e372-432e-9d3e-a5edd741beff'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:49:24.222721 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-c5fee5d1-e372-432e-9d3e-a5edd741beff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '347919e6f5604c62b60e7d5e05a30a20', 'request_id': 'req-c5fee5d1-e372-432e-9d3e-a5edd741beff'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:49:24.234876 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-c5fee5d1-e372-432e-9d3e-a5edd741beff admin admin] The token cache is being invalidate because project 347919e6f5604c62b60e7d5e05a30a20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90653) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:49:24.235119 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-c5fee5d1-e372-432e-9d3e-a5edd741beff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:49:24.235743 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-c5fee5d1-e372-432e-9d3e-a5edd741beff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:49:24.237232 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1089/2182] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:49:24 2022] DELETE /identity/v3/projects/347919e6f5604c62b60e7d5e05a30a20 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:49:31.346049 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0c8765-b293-4684-b541-a30c76fc1c07 service neutron] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:49:31.347557 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-3b0c8765-b293-4684-b541-a30c76fc1c07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:49:31.361795 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0c8765-b293-4684-b541-a30c76fc1c07 service neutron] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:49:31.382372 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:49:31.435782 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacd89eb-b037-42ec-92cc-95300a77e199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:49:31.436431 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd89eb-b037-42ec-92cc-95300a77e199 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:49:31.436708 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd89eb-b037-42ec-92cc-95300a77e199 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:49:31.436982 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd89eb-b037-42ec-92cc-95300a77e199 admin admin] PATH_INFO: `/v3/users/08898a749c3a4be38a452c3fb683a228` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:49:31.438100 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacd89eb-b037-42ec-92cc-95300a77e199 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08898a749c3a4be38a452c3fb683a228)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:49:31.440090 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacd89eb-b037-42ec-92cc-95300a77e199 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:49:31.488212 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-eacd89eb-b037-42ec-92cc-95300a77e199 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08898a749c3a4be38a452c3fb683a228'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:49:31.497632 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-eacd89eb-b037-42ec-92cc-95300a77e199 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08898a749c3a4be38a452c3fb683a228'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:49:31.498898 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-eacd89eb-b037-42ec-92cc-95300a77e199 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08898a749c3a4be38a452c3fb683a228'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:49:31.499564 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-eacd89eb-b037-42ec-92cc-95300a77e199 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08898a749c3a4be38a452c3fb683a228'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:49:31.514827 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-eacd89eb-b037-42ec-92cc-95300a77e199 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08898a749c3a4be38a452c3fb683a228'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:49:31.527693 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1094/2183] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:49:31 2022] DELETE /identity/v3/users/08898a749c3a4be38a452c3fb683a228 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:49:31.532377 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da26a6-6288-40a1-8760-0cb5e9b684d7 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:49:31.532890 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da26a6-6288-40a1-8760-0cb5e9b684d7 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:49:31.532987 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da26a6-6288-40a1-8760-0cb5e9b684d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:49:31.552929 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-b3da26a6-6288-40a1-8760-0cb5e9b684d7 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:49:31.554143 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-b3da26a6-6288-40a1-8760-0cb5e9b684d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:49:31.632891 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1090/2184] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:49:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:49:31.643151 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacd89eb-b037-42ec-92cc-95300a77e199 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:49:31.644655 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-eacd89eb-b037-42ec-92cc-95300a77e199 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:49:31.707911 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacd89eb-b037-42ec-92cc-95300a77e199 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:49:31.711939 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409274b4-abd4-4164-b8cc-bdf78339eb00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:49:31.712615 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409274b4-abd4-4164-b8cc-bdf78339eb00 service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:49:31.713021 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409274b4-abd4-4164-b8cc-bdf78339eb00 service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:49:31.713334 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409274b4-abd4-4164-b8cc-bdf78339eb00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:49:31.724188 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409274b4-abd4-4164-b8cc-bdf78339eb00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:49:31.725518 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409274b4-abd4-4164-b8cc-bdf78339eb00 service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:49:31.736389 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1095/2185] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:49:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:49:32.032222 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3da26a6-6288-40a1-8760-0cb5e9b684d7 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:49:32.034216 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-b3da26a6-6288-40a1-8760-0cb5e9b684d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:49:32.055103 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3da26a6-6288-40a1-8760-0cb5e9b684d7 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:49:32.060886 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7895bf5a-33f4-452e-95ca-68ff6b62b16c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:49:32.061240 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7895bf5a-33f4-452e-95ca-68ff6b62b16c admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:49:32.061525 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7895bf5a-33f4-452e-95ca-68ff6b62b16c admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:49:32.061886 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7895bf5a-33f4-452e-95ca-68ff6b62b16c admin admin] PATH_INFO: `/v3/projects/f82454422ab547af884f7d689ddc9d9f` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:49:32.063222 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7895bf5a-33f4-452e-95ca-68ff6b62b16c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f82454422ab547af884f7d689ddc9d9f)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:49:32.065298 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7895bf5a-33f4-452e-95ca-68ff6b62b16c admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:49:32.094276 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:49:32.125057 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-7895bf5a-33f4-452e-95ca-68ff6b62b16c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f82454422ab547af884f7d689ddc9d9f', 'request_id': 'req-7895bf5a-33f4-452e-95ca-68ff6b62b16c'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:49:32.131648 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-7895bf5a-33f4-452e-95ca-68ff6b62b16c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f82454422ab547af884f7d689ddc9d9f', 'request_id': 'req-7895bf5a-33f4-452e-95ca-68ff6b62b16c'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:49:32.140512 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-7895bf5a-33f4-452e-95ca-68ff6b62b16c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f82454422ab547af884f7d689ddc9d9f', 'request_id': 'req-7895bf5a-33f4-452e-95ca-68ff6b62b16c'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:49:32.152014 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-7895bf5a-33f4-452e-95ca-68ff6b62b16c admin admin] The token cache is being invalidate because project f82454422ab547af884f7d689ddc9d9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90653) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:49:32.152310 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-7895bf5a-33f4-452e-95ca-68ff6b62b16c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:49:32.152962 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-7895bf5a-33f4-452e-95ca-68ff6b62b16c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:49:32.155269 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1091/2186] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:49:32 2022] DELETE /identity/v3/projects/f82454422ab547af884f7d689ddc9d9f => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:49:32.160426 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b0dd2-fe38-4921-9a22-2fb59c099bb0 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:49:32.160733 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b0dd2-fe38-4921-9a22-2fb59c099bb0 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:49:32.161009 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b0dd2-fe38-4921-9a22-2fb59c099bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:49:32.181735 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-926b0dd2-fe38-4921-9a22-2fb59c099bb0 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:49:32.182954 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-926b0dd2-fe38-4921-9a22-2fb59c099bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:49:32.225759 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:49:32.246494 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1096/2187] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:49:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:49:32.254573 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90b6514-9f07-40c6-887a-49ae7f0fe98d None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:49:32.254797 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90b6514-9f07-40c6-887a-49ae7f0fe98d None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:49:32.255072 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90b6514-9f07-40c6-887a-49ae7f0fe98d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:49:32.276180 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-b90b6514-9f07-40c6-887a-49ae7f0fe98d None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:49:32.276885 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-b90b6514-9f07-40c6-887a-49ae7f0fe98d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:49:32.285494 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1092/2188] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:49:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:49:32.289664 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926b0dd2-fe38-4921-9a22-2fb59c099bb0 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:49:32.299657 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926b0dd2-fe38-4921-9a22-2fb59c099bb0 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:49:32.304257 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b617778-f807-4503-87da-23511339dfda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:49:32.304862 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b617778-f807-4503-87da-23511339dfda admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:49:32.305141 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b617778-f807-4503-87da-23511339dfda admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:49:32.305412 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b617778-f807-4503-87da-23511339dfda admin admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:49:32.306059 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b617778-f807-4503-87da-23511339dfda admin admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:49:32.306393 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b617778-f807-4503-87da-23511339dfda admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:49:32.308029 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b617778-f807-4503-87da-23511339dfda admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:49:32.320608 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1097/2189] 10.222.0.15 () {58 vars in 1153 bytes} [Sun Sep 11 10:49:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:49:32.324886 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90b6514-9f07-40c6-887a-49ae7f0fe98d None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:49:32.334612 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90b6514-9f07-40c6-887a-49ae7f0fe98d None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:49:32.339343 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d4a294-48c1-4b66-87c7-a1c90fb44344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:49:32.345812 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d4a294-48c1-4b66-87c7-a1c90fb44344 admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:49:32.346093 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d4a294-48c1-4b66-87c7-a1c90fb44344 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:49:32.346362 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d4a294-48c1-4b66-87c7-a1c90fb44344 admin admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:49:32.347005 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d4a294-48c1-4b66-87c7-a1c90fb44344 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:49:32.348935 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d4a294-48c1-4b66-87c7-a1c90fb44344 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:49:32.385147 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1093/2190] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:49:32 2022] POST /identity/v3/projects => generated 373 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:49:32.389862 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b617778-f807-4503-87da-23511339dfda admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:49:32.400146 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b617778-f807-4503-87da-23511339dfda admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:49:32.469581 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5697b9b2-bf0b-42fc-a059-77c2abe88d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:49:32.470195 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5697b9b2-bf0b-42fc-a059-77c2abe88d1c admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:49:32.470476 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5697b9b2-bf0b-42fc-a059-77c2abe88d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:49:32.470749 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5697b9b2-bf0b-42fc-a059-77c2abe88d1c admin admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:49:32.471561 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5697b9b2-bf0b-42fc-a059-77c2abe88d1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:49:32.473628 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5697b9b2-bf0b-42fc-a059-77c2abe88d1c admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:49:32.496293 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1098/2191] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:49:32 2022] POST /identity/v3/users => generated 287 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:49:32.501153 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d4a294-48c1-4b66-87c7-a1c90fb44344 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:49:32.511765 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d4a294-48c1-4b66-87c7-a1c90fb44344 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:49:32.516673 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0cc7a6-194f-4c27-942b-7eb2b505e91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:49:32.517283 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0cc7a6-194f-4c27-942b-7eb2b505e91d admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:49:32.517564 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0cc7a6-194f-4c27-942b-7eb2b505e91d admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:49:32.517837 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0cc7a6-194f-4c27-942b-7eb2b505e91d admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:49:32.518484 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0cc7a6-194f-4c27-942b-7eb2b505e91d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:49:32.520172 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0cc7a6-194f-4c27-942b-7eb2b505e91d admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:49:32.531536 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1094/2192] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:49:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:49:32.535751 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5697b9b2-bf0b-42fc-a059-77c2abe88d1c admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:49:32.546297 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5697b9b2-bf0b-42fc-a059-77c2abe88d1c admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:49:32.551124 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfd94c1-d97f-41ea-b573-8a8d495f8cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:49:32.551762 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd94c1-d97f-41ea-b573-8a8d495f8cfd admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:49:32.552041 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd94c1-d97f-41ea-b573-8a8d495f8cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:49:32.552313 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd94c1-d97f-41ea-b573-8a8d495f8cfd admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:49:32.552965 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfd94c1-d97f-41ea-b573-8a8d495f8cfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:49:32.554566 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfd94c1-d97f-41ea-b573-8a8d495f8cfd admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:49:32.564949 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1099/2193] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:49:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 10:49:32.569042 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0cc7a6-194f-4c27-942b-7eb2b505e91d admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:49:32.580019 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0cc7a6-194f-4c27-942b-7eb2b505e91d admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:49:32.584781 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f21215-5164-46b7-a898-4502e776b1fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:49:32.585523 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f21215-5164-46b7-a898-4502e776b1fc admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:49:32.585909 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f21215-5164-46b7-a898-4502e776b1fc admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:49:32.586248 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f21215-5164-46b7-a898-4502e776b1fc admin admin] PATH_INFO: `/v3/projects/587882642889489587749d5fb846acc6/users/e839e78f400a46eb916b51c9e257bdf1/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:49:32.601698 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f21215-5164-46b7-a898-4502e776b1fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=587882642889489587749d5fb846acc6, user_id=e839e78f400a46eb916b51c9e257bdf1, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:49:32.604424 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f21215-5164-46b7-a898-4502e776b1fc admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:49:32.618626 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1095/2194] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:49:32 2022] PUT /identity/v3/projects/587882642889489587749d5fb846acc6/users/e839e78f400a46eb916b51c9e257bdf1/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:49:32.623729 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7be5b4-9989-41e4-8719-b12a540212ea None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:49:32.624018 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7be5b4-9989-41e4-8719-b12a540212ea None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:49:32.624288 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7be5b4-9989-41e4-8719-b12a540212ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:49:32.643971 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-bc7be5b4-9989-41e4-8719-b12a540212ea None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:49:32.645189 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-bc7be5b4-9989-41e4-8719-b12a540212ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:49:32.709988 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1100/2195] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:49:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:49:32.720747 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f21215-5164-46b7-a898-4502e776b1fc admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:49:32.722492 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-10f21215-5164-46b7-a898-4502e776b1fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:49:32.785642 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f21215-5164-46b7-a898-4502e776b1fc admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:49:32.789359 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645032f2-cbaf-4079-b951-7847c7be3f45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:49:32.789963 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645032f2-cbaf-4079-b951-7847c7be3f45 service neutron] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:49:32.790247 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645032f2-cbaf-4079-b951-7847c7be3f45 service neutron] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:49:32.790519 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645032f2-cbaf-4079-b951-7847c7be3f45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:49:32.800129 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645032f2-cbaf-4079-b951-7847c7be3f45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:49:32.801551 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645032f2-cbaf-4079-b951-7847c7be3f45 service neutron] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:49:32.810970 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1096/2196] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:49:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:49:36.221593 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7be5b4-9989-41e4-8719-b12a540212ea None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:49:36.223168 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-bc7be5b4-9989-41e4-8719-b12a540212ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:49:36.241152 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7be5b4-9989-41e4-8719-b12a540212ea None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:49:36.247747 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08ea9d8-d081-4faa-8f6e-52933f3e043c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:49:36.249527 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ea9d8-d081-4faa-8f6e-52933f3e043c admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:49:36.252282 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ea9d8-d081-4faa-8f6e-52933f3e043c admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:49:36.253142 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ea9d8-d081-4faa-8f6e-52933f3e043c admin admin] PATH_INFO: `/v3/users/103e0c1384234f4eac3480026c53cbd8` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:49:36.255885 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08ea9d8-d081-4faa-8f6e-52933f3e043c admin admin] RBAC: Authorizing `identity:delete_user(user_id=103e0c1384234f4eac3480026c53cbd8)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:49:36.258661 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08ea9d8-d081-4faa-8f6e-52933f3e043c admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:49:36.371659 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:49:36.374838 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-f08ea9d8-d081-4faa-8f6e-52933f3e043c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '103e0c1384234f4eac3480026c53cbd8'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:49:36.394038 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-f08ea9d8-d081-4faa-8f6e-52933f3e043c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '103e0c1384234f4eac3480026c53cbd8'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:49:36.395648 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-f08ea9d8-d081-4faa-8f6e-52933f3e043c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '103e0c1384234f4eac3480026c53cbd8'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:49:36.396795 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-f08ea9d8-d081-4faa-8f6e-52933f3e043c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '103e0c1384234f4eac3480026c53cbd8'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:49:36.413937 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-f08ea9d8-d081-4faa-8f6e-52933f3e043c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '103e0c1384234f4eac3480026c53cbd8'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:49:36.429540 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1101/2197] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:49:36 2022] DELETE /identity/v3/users/103e0c1384234f4eac3480026c53cbd8 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:49:36.434275 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1626350-52f8-4ba2-8631-6d960ea4e1a3 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:49:36.435888 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1626350-52f8-4ba2-8631-6d960ea4e1a3 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:49:36.436868 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1626350-52f8-4ba2-8631-6d960ea4e1a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:49:36.457438 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-e1626350-52f8-4ba2-8631-6d960ea4e1a3 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:49:36.459069 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-e1626350-52f8-4ba2-8631-6d960ea4e1a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:49:36.506890 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:49:36.557220 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1097/2198] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:49:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:49:36.575560 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08ea9d8-d081-4faa-8f6e-52933f3e043c admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:49:36.578609 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-f08ea9d8-d081-4faa-8f6e-52933f3e043c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:49:36.683462 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08ea9d8-d081-4faa-8f6e-52933f3e043c admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:49:36.688320 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f075aaba-b246-415d-b51e-44eba3cda215 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:49:36.688942 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f075aaba-b246-415d-b51e-44eba3cda215 service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:49:36.689242 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f075aaba-b246-415d-b51e-44eba3cda215 service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:49:36.689506 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f075aaba-b246-415d-b51e-44eba3cda215 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:49:36.719397 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f075aaba-b246-415d-b51e-44eba3cda215 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:49:36.721234 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f075aaba-b246-415d-b51e-44eba3cda215 service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:49:36.750743 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1102/2199] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:49:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:49:37.115519 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1626350-52f8-4ba2-8631-6d960ea4e1a3 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:49:37.117171 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-e1626350-52f8-4ba2-8631-6d960ea4e1a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:49:37.135315 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1626350-52f8-4ba2-8631-6d960ea4e1a3 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:49:37.141562 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb7e37e-afdc-410f-8bf2-6e975c3ab4a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:49:37.142448 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb7e37e-afdc-410f-8bf2-6e975c3ab4a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:49:37.142882 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb7e37e-afdc-410f-8bf2-6e975c3ab4a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:49:37.143382 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb7e37e-afdc-410f-8bf2-6e975c3ab4a2 admin admin] PATH_INFO: `/v3/projects/45a50e1025d949cb8c59366b5646d7b6` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:49:37.144760 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb7e37e-afdc-410f-8bf2-6e975c3ab4a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45a50e1025d949cb8c59366b5646d7b6)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:49:37.147021 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb7e37e-afdc-410f-8bf2-6e975c3ab4a2 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:49:37.229201 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-adb7e37e-afdc-410f-8bf2-6e975c3ab4a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45a50e1025d949cb8c59366b5646d7b6', 'request_id': 'req-adb7e37e-afdc-410f-8bf2-6e975c3ab4a2'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:49:37.236921 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-adb7e37e-afdc-410f-8bf2-6e975c3ab4a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45a50e1025d949cb8c59366b5646d7b6', 'request_id': 'req-adb7e37e-afdc-410f-8bf2-6e975c3ab4a2'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:49:37.247053 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-adb7e37e-afdc-410f-8bf2-6e975c3ab4a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45a50e1025d949cb8c59366b5646d7b6', 'request_id': 'req-adb7e37e-afdc-410f-8bf2-6e975c3ab4a2'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:49:37.261495 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-adb7e37e-afdc-410f-8bf2-6e975c3ab4a2 admin admin] The token cache is being invalidate because project 45a50e1025d949cb8c59366b5646d7b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90653) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:49:37.261823 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-adb7e37e-afdc-410f-8bf2-6e975c3ab4a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:49:37.262446 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-adb7e37e-afdc-410f-8bf2-6e975c3ab4a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:49:37.266177 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1098/2200] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:49:37 2022] DELETE /identity/v3/projects/45a50e1025d949cb8c59366b5646d7b6 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:49:37.270841 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256b5b91-d17f-4d40-9b4c-39b16fa681c6 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:49:37.271344 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256b5b91-d17f-4d40-9b4c-39b16fa681c6 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:49:37.271487 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256b5b91-d17f-4d40-9b4c-39b16fa681c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:49:37.297108 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-256b5b91-d17f-4d40-9b4c-39b16fa681c6 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:49:37.298398 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-256b5b91-d17f-4d40-9b4c-39b16fa681c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:49:37.382860 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1103/2201] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:49:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 10:49:37.392198 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147bfef1-d434-47c9-8116-7a9593bab1f9 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:49:37.392488 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147bfef1-d434-47c9-8116-7a9593bab1f9 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:49:37.392764 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147bfef1-d434-47c9-8116-7a9593bab1f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:49:37.414930 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-147bfef1-d434-47c9-8116-7a9593bab1f9 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:49:37.416363 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-147bfef1-d434-47c9-8116-7a9593bab1f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:49:37.430488 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256b5b91-d17f-4d40-9b4c-39b16fa681c6 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:49:37.436580 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1099/2202] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:49:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:49:37.442133 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256b5b91-d17f-4d40-9b4c-39b16fa681c6 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:49:37.447594 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafc28fe-8916-4f55-a03b-e92c727efb24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:49:37.448312 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafc28fe-8916-4f55-a03b-e92c727efb24 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:49:37.448668 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafc28fe-8916-4f55-a03b-e92c727efb24 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:49:37.449007 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafc28fe-8916-4f55-a03b-e92c727efb24 admin admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:49:37.449738 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafc28fe-8916-4f55-a03b-e92c727efb24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:49:37.450132 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafc28fe-8916-4f55-a03b-e92c727efb24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:49:37.452187 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafc28fe-8916-4f55-a03b-e92c727efb24 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:49:37.466083 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1104/2203] 10.222.0.15 () {58 vars in 1153 bytes} [Sun Sep 11 10:49:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:49:37.470416 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147bfef1-d434-47c9-8116-7a9593bab1f9 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:49:37.481220 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147bfef1-d434-47c9-8116-7a9593bab1f9 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:49:37.488223 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560a0248-d359-412f-a89e-eeba6d6c1f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:49:37.489135 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a0248-d359-412f-a89e-eeba6d6c1f81 admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:49:37.489614 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a0248-d359-412f-a89e-eeba6d6c1f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:49:37.490035 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a0248-d359-412f-a89e-eeba6d6c1f81 admin admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:49:37.490847 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560a0248-d359-412f-a89e-eeba6d6c1f81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:49:37.493800 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560a0248-d359-412f-a89e-eeba6d6c1f81 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:49:37.539931 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1100/2204] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:49:37 2022] POST /identity/v3/projects => generated 391 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:49:37.545068 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafc28fe-8916-4f55-a03b-e92c727efb24 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:49:37.555554 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafc28fe-8916-4f55-a03b-e92c727efb24 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:49:37.616698 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:49:37.636142 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f95d2d8-c9b2-4ac2-8558-19b346bdc0ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:49:37.637203 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f95d2d8-c9b2-4ac2-8558-19b346bdc0ef admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:49:37.637964 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f95d2d8-c9b2-4ac2-8558-19b346bdc0ef admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:49:37.638796 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f95d2d8-c9b2-4ac2-8558-19b346bdc0ef admin admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:49:37.639844 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f95d2d8-c9b2-4ac2-8558-19b346bdc0ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:49:37.642407 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f95d2d8-c9b2-4ac2-8558-19b346bdc0ef admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:49:37.681789 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1105/2205] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:49:37 2022] POST /identity/v3/users => generated 296 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:49:37.686073 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560a0248-d359-412f-a89e-eeba6d6c1f81 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:49:37.706008 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560a0248-d359-412f-a89e-eeba6d6c1f81 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:49:37.712609 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533a6afd-a2fa-4575-8961-865d93a93e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:49:37.714611 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533a6afd-a2fa-4575-8961-865d93a93e9c admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:49:37.715187 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533a6afd-a2fa-4575-8961-865d93a93e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:49:37.715841 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533a6afd-a2fa-4575-8961-865d93a93e9c admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:49:37.726555 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0cc9c3-9654-4c2b-8b36-daa82c1a17f8 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:49:37.726897 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0cc9c3-9654-4c2b-8b36-daa82c1a17f8 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:49:37.727118 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0cc9c3-9654-4c2b-8b36-daa82c1a17f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:49:37.734350 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533a6afd-a2fa-4575-8961-865d93a93e9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:49:37.736304 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533a6afd-a2fa-4575-8961-865d93a93e9c admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:49:37.753354 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-2b0cc9c3-9654-4c2b-8b36-daa82c1a17f8 None None] MFA Rules not processed for user `30f5e5a2ce8749d692600f1d2ab4fdb9`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:49:37.754654 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-2b0cc9c3-9654-4c2b-8b36-daa82c1a17f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:49:37.758538 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1101/2206] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:49:37 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:49:37.762149 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533a6afd-a2fa-4575-8961-865d93a93e9c admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:49:37.782451 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533a6afd-a2fa-4575-8961-865d93a93e9c admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:49:37.799865 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0c570c-b632-4516-a254-63acb55d38dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:49:37.800490 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0c570c-b632-4516-a254-63acb55d38dc admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:49:37.800777 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0c570c-b632-4516-a254-63acb55d38dc admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:49:37.829053 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0c570c-b632-4516-a254-63acb55d38dc admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:49:37.829938 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0c570c-b632-4516-a254-63acb55d38dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:49:37.834548 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0c570c-b632-4516-a254-63acb55d38dc admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:49:37.849426 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1106/2207] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:49:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:49:37.859478 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1102/2208] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:49:37 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 10:49:37.861118 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0cc9c3-9654-4c2b-8b36-daa82c1a17f8 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:49:37.876879 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0cc9c3-9654-4c2b-8b36-daa82c1a17f8 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:49:37.883716 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab15a79-dc3f-44b6-9193-4cc3bffc677b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:49:37.884948 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab15a79-dc3f-44b6-9193-4cc3bffc677b admin admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:49:37.885265 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab15a79-dc3f-44b6-9193-4cc3bffc677b admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:49:37.885546 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab15a79-dc3f-44b6-9193-4cc3bffc677b admin admin] PATH_INFO: `/v3/projects/668ab169184947d69e5f713cbfee5f76/users/73048dd7e2594b24bdd209facba93031/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:49:37.932651 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab15a79-dc3f-44b6-9193-4cc3bffc677b admin admin] RBAC: Authorizing `identity:create_grant(project_id=668ab169184947d69e5f713cbfee5f76, user_id=73048dd7e2594b24bdd209facba93031, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:49:37.937208 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab15a79-dc3f-44b6-9193-4cc3bffc677b admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:49:37.983800 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b4f80a-f585-4dba-be4a-89b2b367333c None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:49:37.984181 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b4f80a-f585-4dba-be4a-89b2b367333c None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:49:37.984442 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b4f80a-f585-4dba-be4a-89b2b367333c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:49:37.987492 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1107/2209] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:49:37 2022] PUT /identity/v3/projects/668ab169184947d69e5f713cbfee5f76/users/73048dd7e2594b24bdd209facba93031/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:49:38.019407 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-20b4f80a-f585-4dba-be4a-89b2b367333c None None] MFA Rules not processed for user `73048dd7e2594b24bdd209facba93031`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:49:38.022610 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-20b4f80a-f585-4dba-be4a-89b2b367333c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:49:38.129443 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1103/2210] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:49:37 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:49:38.140026 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab15a79-dc3f-44b6-9193-4cc3bffc677b admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:49:38.141827 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-dab15a79-dc3f-44b6-9193-4cc3bffc677b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:49:38.264534 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab15a79-dc3f-44b6-9193-4cc3bffc677b admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:49:38.268501 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa22ba-c937-4c03-8f53-cb1600602389 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:49:38.269278 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa22ba-c937-4c03-8f53-cb1600602389 service cinder] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:49:38.270911 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa22ba-c937-4c03-8f53-cb1600602389 service cinder] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:49:38.270911 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa22ba-c937-4c03-8f53-cb1600602389 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:49:38.281256 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fa22ba-c937-4c03-8f53-cb1600602389 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:49:38.282723 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fa22ba-c937-4c03-8f53-cb1600602389 service cinder] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:49:38.296214 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1108/2211] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:49:38 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:49:39.267251 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c0a76a-3047-4272-9e1c-0f83ba494207 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:49:39.267780 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c0a76a-3047-4272-9e1c-0f83ba494207 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:49:39.267908 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c0a76a-3047-4272-9e1c-0f83ba494207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:49:39.300103 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:49:39.319334 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-b5c0a76a-3047-4272-9e1c-0f83ba494207 None None] MFA Rules not processed for user `e839e78f400a46eb916b51c9e257bdf1`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:49:39.320937 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-b5c0a76a-3047-4272-9e1c-0f83ba494207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:49:39.406061 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1104/2212] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:49:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:49:39.410684 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa22ba-c937-4c03-8f53-cb1600602389 service cinder] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:49:39.412163 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-c3fa22ba-c937-4c03-8f53-cb1600602389 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:49:39.487014 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa22ba-c937-4c03-8f53-cb1600602389 service cinder] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:49:39.491598 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6584c3d9-6517-4124-8ccb-bcadb6ffb169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:49:39.492198 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6584c3d9-6517-4124-8ccb-bcadb6ffb169 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:49:39.492475 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6584c3d9-6517-4124-8ccb-bcadb6ffb169 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:49:39.492745 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6584c3d9-6517-4124-8ccb-bcadb6ffb169 admin admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:49:39.493365 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6584c3d9-6517-4124-8ccb-bcadb6ffb169 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:49:39.495257 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6584c3d9-6517-4124-8ccb-bcadb6ffb169 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:49:39.507464 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:49:39.525623 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1109/2213] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:49:39 2022] POST /identity/v3/projects => generated 373 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:49:39.529577 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c0a76a-3047-4272-9e1c-0f83ba494207 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:49:39.540672 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c0a76a-3047-4272-9e1c-0f83ba494207 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:49:39.604236 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634f5d3e-e8c7-4450-b8b7-3c3b03049c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:49:39.604859 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634f5d3e-e8c7-4450-b8b7-3c3b03049c2e admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:49:39.605131 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634f5d3e-e8c7-4450-b8b7-3c3b03049c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:49:39.605403 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634f5d3e-e8c7-4450-b8b7-3c3b03049c2e admin admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:49:39.606065 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634f5d3e-e8c7-4450-b8b7-3c3b03049c2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:49:39.608341 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634f5d3e-e8c7-4450-b8b7-3c3b03049c2e admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:49:39.635326 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1105/2214] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:49:39 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:49:39.643347 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6584c3d9-6517-4124-8ccb-bcadb6ffb169 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:49:39.661683 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6584c3d9-6517-4124-8ccb-bcadb6ffb169 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:49:39.668704 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce67fc82-652c-47c1-8df2-f2b080824074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:49:39.670250 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce67fc82-652c-47c1-8df2-f2b080824074 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:49:39.670250 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce67fc82-652c-47c1-8df2-f2b080824074 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:49:39.670506 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce67fc82-652c-47c1-8df2-f2b080824074 admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:49:39.671342 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce67fc82-652c-47c1-8df2-f2b080824074 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:49:39.673474 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce67fc82-652c-47c1-8df2-f2b080824074 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:49:39.697178 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634f5d3e-e8c7-4450-b8b7-3c3b03049c2e admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:49:39.707478 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1110/2215] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:49:39 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:49:39.709845 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634f5d3e-e8c7-4450-b8b7-3c3b03049c2e admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:49:39.717114 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19664c0e-dae6-4513-aead-eaabd44b7842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:49:39.717806 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19664c0e-dae6-4513-aead-eaabd44b7842 admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:49:39.718271 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19664c0e-dae6-4513-aead-eaabd44b7842 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:49:39.718553 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19664c0e-dae6-4513-aead-eaabd44b7842 admin admin] PATH_INFO: `/v3/projects/642eea5bb9754514a78a7d16660332ee/users/2eb2819aee9646d692373511ed762e8a/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:49:39.737002 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19664c0e-dae6-4513-aead-eaabd44b7842 admin admin] RBAC: Authorizing `identity:create_grant(project_id=642eea5bb9754514a78a7d16660332ee, user_id=2eb2819aee9646d692373511ed762e8a, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:49:39.742371 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19664c0e-dae6-4513-aead-eaabd44b7842 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:49:39.770837 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce67fc82-652c-47c1-8df2-f2b080824074 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:49:39.772181 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1106/2216] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:49:39 2022] PUT /identity/v3/projects/642eea5bb9754514a78a7d16660332ee/users/2eb2819aee9646d692373511ed762e8a/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:49:39.804295 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce67fc82-652c-47c1-8df2-f2b080824074 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:49:39.934068 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59c67f1-1354-4772-a90e-c6bc55c12cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:49:39.936064 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59c67f1-1354-4772-a90e-c6bc55c12cf9 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:49:39.936367 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59c67f1-1354-4772-a90e-c6bc55c12cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:49:39.937006 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59c67f1-1354-4772-a90e-c6bc55c12cf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:49:39.939275 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59c67f1-1354-4772-a90e-c6bc55c12cf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:49:39.943131 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59c67f1-1354-4772-a90e-c6bc55c12cf9 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:49:39.959042 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1111/2217] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:49:39 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:49:39.963204 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19664c0e-dae6-4513-aead-eaabd44b7842 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:49:39.980052 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19664c0e-dae6-4513-aead-eaabd44b7842 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:49:40.014076 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b821807-e777-4d30-b63f-307c7981d424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:49:40.019316 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b821807-e777-4d30-b63f-307c7981d424 admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:49:40.019662 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b821807-e777-4d30-b63f-307c7981d424 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:49:40.020162 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b821807-e777-4d30-b63f-307c7981d424 admin admin] PATH_INFO: `/v3/projects/642eea5bb9754514a78a7d16660332ee/users/2eb2819aee9646d692373511ed762e8a/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:49:40.024756 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b821807-e777-4d30-b63f-307c7981d424 admin admin] RBAC: Authorizing `identity:create_grant(project_id=642eea5bb9754514a78a7d16660332ee, user_id=2eb2819aee9646d692373511ed762e8a, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:49:40.029742 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b821807-e777-4d30-b63f-307c7981d424 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:49:40.063625 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1107/2218] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:49:39 2022] PUT /identity/v3/projects/642eea5bb9754514a78a7d16660332ee/users/2eb2819aee9646d692373511ed762e8a/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:49:44.445311 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f532680a-b1a5-40c3-b7e0-3166931d9c5e None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:49:44.445864 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f532680a-b1a5-40c3-b7e0-3166931d9c5e None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:49:44.446009 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f532680a-b1a5-40c3-b7e0-3166931d9c5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:49:44.475244 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-f532680a-b1a5-40c3-b7e0-3166931d9c5e None None] MFA Rules not processed for user `2eb2819aee9646d692373511ed762e8a`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:49:44.476500 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-f532680a-b1a5-40c3-b7e0-3166931d9c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:49:44.548480 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1112/2219] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:49:44 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 10:49:44.562796 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b821807-e777-4d30-b63f-307c7981d424 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:49:44.565089 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-7b821807-e777-4d30-b63f-307c7981d424 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:49:44.630616 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:49:44.657805 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b821807-e777-4d30-b63f-307c7981d424 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:49:44.664506 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad8c5fc-c7e0-4e3f-ae54-103857f97bf4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bd5356829b6454993bab9bad2f79fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:49:44.665336 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad8c5fc-c7e0-4e3f-ae54-103857f97bf4 service nova] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:49:44.666178 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad8c5fc-c7e0-4e3f-ae54-103857f97bf4 service nova] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:49:44.666178 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad8c5fc-c7e0-4e3f-ae54-103857f97bf4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:49:44.752510 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad8c5fc-c7e0-4e3f-ae54-103857f97bf4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:49:44.754150 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad8c5fc-c7e0-4e3f-ae54-103857f97bf4 service nova] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:49:44.767298 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1108/2220] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:49:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:49:46.802419 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfa4206-1af3-410f-bdd7-3e050253c2b2 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:49:46.802932 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfa4206-1af3-410f-bdd7-3e050253c2b2 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:49:46.803178 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfa4206-1af3-410f-bdd7-3e050253c2b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:49:46.822833 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-6dfa4206-1af3-410f-bdd7-3e050253c2b2 None None] MFA Rules not processed for user `e839e78f400a46eb916b51c9e257bdf1`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:49:46.824176 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-6dfa4206-1af3-410f-bdd7-3e050253c2b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:49:46.832338 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1113/2221] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:49:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:49:48.491040 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad8c5fc-c7e0-4e3f-ae54-103857f97bf4 service nova] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:49:48.492800 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-7ad8c5fc-c7e0-4e3f-ae54-103857f97bf4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:49:48.568621 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad8c5fc-c7e0-4e3f-ae54-103857f97bf4 service nova] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:49:48.574485 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4eaceb-4638-4964-af5a-dd3dfd28aa97 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08477b4c88f04c9e97391aba401879e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:49:48.575220 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4eaceb-4638-4964-af5a-dd3dfd28aa97 service placement] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:49:48.575637 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4eaceb-4638-4964-af5a-dd3dfd28aa97 service placement] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:49:48.576166 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4eaceb-4638-4964-af5a-dd3dfd28aa97 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:49:48.578072 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-4d4eaceb-4638-4964-af5a-dd3dfd28aa97 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:49:48.590554 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4eaceb-4638-4964-af5a-dd3dfd28aa97 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:49:48.593161 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4eaceb-4638-4964-af5a-dd3dfd28aa97 service placement] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:49:48.605733 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1109/2222] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:49:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:49:56.817560 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=74,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:49:58.601079 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=65 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:49:59.523693 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4098b5-ec1e-4737-b13e-1e292e1fbe49 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:49:59.524179 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4098b5-ec1e-4737-b13e-1e292e1fbe49 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:49:59.524294 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4098b5-ec1e-4737-b13e-1e292e1fbe49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:49:59.543472 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-bc4098b5-ec1e-4737-b13e-1e292e1fbe49 None None] MFA Rules not processed for user `f6eaba3a278e4bc99aca0dedd649fa9f`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:49:59.545232 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-bc4098b5-ec1e-4737-b13e-1e292e1fbe49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:49:59.598931 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1114/2223] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:49:59 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:50:09.592266 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=25 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:50:15.629522 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4eaceb-4638-4964-af5a-dd3dfd28aa97 service placement] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:50:15.693684 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4eaceb-4638-4964-af5a-dd3dfd28aa97 service placement] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:50:15.698182 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded2738d-456b-47e9-9b06-468696593b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:50:15.698805 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded2738d-456b-47e9-9b06-468696593b03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:15.699086 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded2738d-456b-47e9-9b06-468696593b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:15.699360 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded2738d-456b-47e9-9b06-468696593b03 admin admin] PATH_INFO: `/v3/users/73048dd7e2594b24bdd209facba93031` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:15.700497 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded2738d-456b-47e9-9b06-468696593b03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73048dd7e2594b24bdd209facba93031)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:50:15.702778 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded2738d-456b-47e9-9b06-468696593b03 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:50:15.749692 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-ded2738d-456b-47e9-9b06-468696593b03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73048dd7e2594b24bdd209facba93031'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:50:15.757386 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-ded2738d-456b-47e9-9b06-468696593b03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73048dd7e2594b24bdd209facba93031'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:50:15.758052 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-ded2738d-456b-47e9-9b06-468696593b03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73048dd7e2594b24bdd209facba93031'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:50:15.758663 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-ded2738d-456b-47e9-9b06-468696593b03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73048dd7e2594b24bdd209facba93031'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:50:15.765948 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-ded2738d-456b-47e9-9b06-468696593b03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73048dd7e2594b24bdd209facba93031'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:50:15.784421 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1110/2224] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:50:15 2022] DELETE /identity/v3/users/73048dd7e2594b24bdd209facba93031 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:50:15.788873 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be19b95b-52a5-4026-b3e1-4604aceae37e None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:15.789358 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be19b95b-52a5-4026-b3e1-4604aceae37e None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:15.789459 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be19b95b-52a5-4026-b3e1-4604aceae37e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:15.808631 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-be19b95b-52a5-4026-b3e1-4604aceae37e None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:50:15.809820 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-be19b95b-52a5-4026-b3e1-4604aceae37e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:50:15.872059 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1115/2225] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:50:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:50:15.882285 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded2738d-456b-47e9-9b06-468696593b03 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:50:15.883739 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-ded2738d-456b-47e9-9b06-468696593b03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:50:15.943403 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded2738d-456b-47e9-9b06-468696593b03 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:50:15.947334 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb6ddef-f8b3-4670-801e-d60907b293e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:50:15.947978 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb6ddef-f8b3-4670-801e-d60907b293e4 service neutron] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:15.948375 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb6ddef-f8b3-4670-801e-d60907b293e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:15.948648 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb6ddef-f8b3-4670-801e-d60907b293e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:15.958840 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb6ddef-f8b3-4670-801e-d60907b293e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:50:15.960285 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb6ddef-f8b3-4670-801e-d60907b293e4 service neutron] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:50:15.970630 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1111/2226] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:50:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:50:16.304776 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be19b95b-52a5-4026-b3e1-4604aceae37e None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:50:16.306315 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-be19b95b-52a5-4026-b3e1-4604aceae37e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:50:16.320174 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be19b95b-52a5-4026-b3e1-4604aceae37e None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:50:16.325555 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a252cf53-93f6-4b46-bc4a-cec7f01e833e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:50:16.326877 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a252cf53-93f6-4b46-bc4a-cec7f01e833e admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:16.327163 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a252cf53-93f6-4b46-bc4a-cec7f01e833e admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:16.327465 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a252cf53-93f6-4b46-bc4a-cec7f01e833e admin admin] PATH_INFO: `/v3/projects/668ab169184947d69e5f713cbfee5f76` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:16.328633 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a252cf53-93f6-4b46-bc4a-cec7f01e833e admin admin] RBAC: Authorizing `identity:delete_project(project_id=668ab169184947d69e5f713cbfee5f76)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:50:16.330638 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a252cf53-93f6-4b46-bc4a-cec7f01e833e admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:50:16.377378 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-a252cf53-93f6-4b46-bc4a-cec7f01e833e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '668ab169184947d69e5f713cbfee5f76', 'request_id': 'req-a252cf53-93f6-4b46-bc4a-cec7f01e833e'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:50:16.383547 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-a252cf53-93f6-4b46-bc4a-cec7f01e833e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '668ab169184947d69e5f713cbfee5f76', 'request_id': 'req-a252cf53-93f6-4b46-bc4a-cec7f01e833e'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:50:16.389917 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-a252cf53-93f6-4b46-bc4a-cec7f01e833e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '668ab169184947d69e5f713cbfee5f76', 'request_id': 'req-a252cf53-93f6-4b46-bc4a-cec7f01e833e'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:50:16.401267 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-a252cf53-93f6-4b46-bc4a-cec7f01e833e admin admin] The token cache is being invalidate because project 668ab169184947d69e5f713cbfee5f76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:50:16.401556 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-a252cf53-93f6-4b46-bc4a-cec7f01e833e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:50:16.402180 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-a252cf53-93f6-4b46-bc4a-cec7f01e833e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:50:16.403690 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1116/2227] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:50:16 2022] DELETE /identity/v3/projects/668ab169184947d69e5f713cbfee5f76 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:50:16.409425 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73085dba-d4ee-4270-9181-1728e5eb67de None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:16.409717 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73085dba-d4ee-4270-9181-1728e5eb67de None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:16.409997 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73085dba-d4ee-4270-9181-1728e5eb67de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:16.429079 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-73085dba-d4ee-4270-9181-1728e5eb67de None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:50:16.430265 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-73085dba-d4ee-4270-9181-1728e5eb67de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:50:16.446571 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:50:16.493303 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1112/2228] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:50:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 10:50:16.502385 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc74a623-08f5-478d-a4a1-c280abed0976 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:16.502678 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc74a623-08f5-478d-a4a1-c280abed0976 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:16.503016 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc74a623-08f5-478d-a4a1-c280abed0976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:16.524919 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-fc74a623-08f5-478d-a4a1-c280abed0976 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:50:16.526415 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-fc74a623-08f5-478d-a4a1-c280abed0976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:50:16.535104 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1117/2229] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:50:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:50:16.539391 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73085dba-d4ee-4270-9181-1728e5eb67de None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:50:16.550974 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73085dba-d4ee-4270-9181-1728e5eb67de None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:50:16.555870 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6126d0-ed9b-4f5a-a40d-4556073d6d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:50:16.556473 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6126d0-ed9b-4f5a-a40d-4556073d6d3e admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:16.556873 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6126d0-ed9b-4f5a-a40d-4556073d6d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:16.557150 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6126d0-ed9b-4f5a-a40d-4556073d6d3e admin admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:16.557793 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6126d0-ed9b-4f5a-a40d-4556073d6d3e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:50:16.558091 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6126d0-ed9b-4f5a-a40d-4556073d6d3e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:50:16.559718 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6126d0-ed9b-4f5a-a40d-4556073d6d3e admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:50:16.577058 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1113/2230] 10.222.0.15 () {58 vars in 1153 bytes} [Sun Sep 11 10:50:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:50:16.578540 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc74a623-08f5-478d-a4a1-c280abed0976 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:50:16.588162 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc74a623-08f5-478d-a4a1-c280abed0976 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:50:16.593162 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a009cc0-ff84-44fd-af01-c87061f33764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:50:16.593899 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a009cc0-ff84-44fd-af01-c87061f33764 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:16.594301 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a009cc0-ff84-44fd-af01-c87061f33764 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:16.594647 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a009cc0-ff84-44fd-af01-c87061f33764 admin admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:16.595355 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a009cc0-ff84-44fd-af01-c87061f33764 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:50:16.597387 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a009cc0-ff84-44fd-af01-c87061f33764 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:50:16.634752 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1118/2231] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:50:16 2022] POST /identity/v3/projects => generated 377 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:50:16.638674 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6126d0-ed9b-4f5a-a40d-4556073d6d3e admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:50:16.654709 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6126d0-ed9b-4f5a-a40d-4556073d6d3e admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:50:16.731767 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda7ddbf-16b4-4d77-a348-6ce08dfeb744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:50:16.732686 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda7ddbf-16b4-4d77-a348-6ce08dfeb744 admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:16.732984 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda7ddbf-16b4-4d77-a348-6ce08dfeb744 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:16.733261 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda7ddbf-16b4-4d77-a348-6ce08dfeb744 admin admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:16.733899 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda7ddbf-16b4-4d77-a348-6ce08dfeb744 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:50:16.735856 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda7ddbf-16b4-4d77-a348-6ce08dfeb744 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:50:16.761608 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1114/2232] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:50:16 2022] POST /identity/v3/users => generated 289 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 10:50:16.763048 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a009cc0-ff84-44fd-af01-c87061f33764 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:50:16.774740 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a009cc0-ff84-44fd-af01-c87061f33764 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:50:16.785004 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f9fa1a-3814-4a21-b17d-8483feec0611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:50:16.785660 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f9fa1a-3814-4a21-b17d-8483feec0611 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:16.785943 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f9fa1a-3814-4a21-b17d-8483feec0611 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:16.786217 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f9fa1a-3814-4a21-b17d-8483feec0611 admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:16.786860 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f9fa1a-3814-4a21-b17d-8483feec0611 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:50:16.788511 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f9fa1a-3814-4a21-b17d-8483feec0611 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:50:16.799521 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1119/2233] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:50:16 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:50:16.803446 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda7ddbf-16b4-4d77-a348-6ce08dfeb744 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:50:16.813480 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda7ddbf-16b4-4d77-a348-6ce08dfeb744 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:50:16.818481 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34d2c72-b874-435e-8a48-5e61803b1abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:50:16.819075 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34d2c72-b874-435e-8a48-5e61803b1abd admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:16.819354 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34d2c72-b874-435e-8a48-5e61803b1abd admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:16.819664 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34d2c72-b874-435e-8a48-5e61803b1abd admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:16.820295 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34d2c72-b874-435e-8a48-5e61803b1abd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:50:16.821931 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34d2c72-b874-435e-8a48-5e61803b1abd admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:50:16.832364 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1115/2234] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:50:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:50:16.836352 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f9fa1a-3814-4a21-b17d-8483feec0611 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:50:16.846627 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f9fa1a-3814-4a21-b17d-8483feec0611 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:50:16.851160 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d924e91-28ca-4e24-aa35-88d2191c963a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:50:16.851839 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d924e91-28ca-4e24-aa35-88d2191c963a admin admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:16.852120 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d924e91-28ca-4e24-aa35-88d2191c963a admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:16.852395 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d924e91-28ca-4e24-aa35-88d2191c963a admin admin] PATH_INFO: `/v3/projects/23556239691f46aa8b39bb6219fc591f/users/e1b0815853b947bc8f50d91c1ce0ab5a/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:16.867896 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d924e91-28ca-4e24-aa35-88d2191c963a admin admin] RBAC: Authorizing `identity:create_grant(project_id=23556239691f46aa8b39bb6219fc591f, user_id=e1b0815853b947bc8f50d91c1ce0ab5a, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:50:16.870531 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d924e91-28ca-4e24-aa35-88d2191c963a admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:50:16.890823 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1120/2235] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:50:16 2022] PUT /identity/v3/projects/23556239691f46aa8b39bb6219fc591f/users/e1b0815853b947bc8f50d91c1ce0ab5a/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 10:50:16.895975 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1192390e-e1d2-49cc-9d04-839aaf694be2 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:16.896439 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1192390e-e1d2-49cc-9d04-839aaf694be2 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:16.896742 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1192390e-e1d2-49cc-9d04-839aaf694be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:16.916011 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-1192390e-e1d2-49cc-9d04-839aaf694be2 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:50:16.917213 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-1192390e-e1d2-49cc-9d04-839aaf694be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:50:16.976358 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1116/2236] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:50:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 10:50:16.986358 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d924e91-28ca-4e24-aa35-88d2191c963a admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:50:16.988112 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-5d924e91-28ca-4e24-aa35-88d2191c963a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:50:17.009940 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:50:17.046220 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d924e91-28ca-4e24-aa35-88d2191c963a admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:50:17.050054 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7489e2a-fa5d-47be-b27c-0203c7c28c10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:50:17.050660 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7489e2a-fa5d-47be-b27c-0203c7c28c10 service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:17.050948 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7489e2a-fa5d-47be-b27c-0203c7c28c10 service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:17.051223 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7489e2a-fa5d-47be-b27c-0203c7c28c10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:17.060238 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7489e2a-fa5d-47be-b27c-0203c7c28c10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:50:17.062372 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7489e2a-fa5d-47be-b27c-0203c7c28c10 service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:50:17.071774 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1121/2237] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:50:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:50:21.687703 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe415070-ac31-426c-800c-9d7045f8d164 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:21.688125 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe415070-ac31-426c-800c-9d7045f8d164 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:21.688218 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe415070-ac31-426c-800c-9d7045f8d164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:21.720759 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-fe415070-ac31-426c-800c-9d7045f8d164 None None] MFA Rules not processed for user `e1b0815853b947bc8f50d91c1ce0ab5a`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:50:21.722087 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-fe415070-ac31-426c-800c-9d7045f8d164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:50:21.746065 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:50:21.851454 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1117/2238] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:50:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:50:21.854022 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7489e2a-fa5d-47be-b27c-0203c7c28c10 service neutron] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:50:21.865099 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7489e2a-fa5d-47be-b27c-0203c7c28c10 service neutron] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:50:21.870446 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b27886-92be-47b1-8b30-5cdd688daa63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:50:21.871392 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b27886-92be-47b1-8b30-5cdd688daa63 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:21.871962 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b27886-92be-47b1-8b30-5cdd688daa63 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:21.872455 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b27886-92be-47b1-8b30-5cdd688daa63 admin admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:21.873340 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b27886-92be-47b1-8b30-5cdd688daa63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:50:21.876166 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b27886-92be-47b1-8b30-5cdd688daa63 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:50:21.923512 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1122/2239] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:50:21 2022] POST /identity/v3/projects => generated 379 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:50:21.927503 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe415070-ac31-426c-800c-9d7045f8d164 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:50:21.954315 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe415070-ac31-426c-800c-9d7045f8d164 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:50:22.028253 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68418ce9-07ac-4547-818e-91d651f8e1ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:50:22.028956 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68418ce9-07ac-4547-818e-91d651f8e1ef admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:22.029347 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68418ce9-07ac-4547-818e-91d651f8e1ef admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:22.029721 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68418ce9-07ac-4547-818e-91d651f8e1ef admin admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:22.030439 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68418ce9-07ac-4547-818e-91d651f8e1ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:50:22.033153 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68418ce9-07ac-4547-818e-91d651f8e1ef admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:50:22.058279 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1118/2240] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:50:21 2022] POST /identity/v3/users => generated 296 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:50:22.063804 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b27886-92be-47b1-8b30-5cdd688daa63 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:50:22.074647 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b27886-92be-47b1-8b30-5cdd688daa63 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:50:22.080290 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8e1e12-19b3-4272-9c25-a3a000109a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:50:22.080918 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8e1e12-19b3-4272-9c25-a3a000109a4f admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:22.081387 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8e1e12-19b3-4272-9c25-a3a000109a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:22.081668 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8e1e12-19b3-4272-9c25-a3a000109a4f admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:22.082322 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8e1e12-19b3-4272-9c25-a3a000109a4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:50:22.084301 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8e1e12-19b3-4272-9c25-a3a000109a4f admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:50:22.097878 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1123/2241] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:50:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:50:22.102237 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68418ce9-07ac-4547-818e-91d651f8e1ef admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:50:22.114122 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68418ce9-07ac-4547-818e-91d651f8e1ef admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:50:22.119997 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d50f95-8c64-49b7-a565-630f7a78c3f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:50:22.120737 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d50f95-8c64-49b7-a565-630f7a78c3f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:22.121102 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d50f95-8c64-49b7-a565-630f7a78c3f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:22.121386 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d50f95-8c64-49b7-a565-630f7a78c3f7 admin admin] PATH_INFO: `/v3/projects/2ded14adc3704a139c3e310030b1f5bc/users/bbd3041dc72e4db29a7eccb276e00de4/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:22.139630 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d50f95-8c64-49b7-a565-630f7a78c3f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ded14adc3704a139c3e310030b1f5bc, user_id=bbd3041dc72e4db29a7eccb276e00de4, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:50:22.142635 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d50f95-8c64-49b7-a565-630f7a78c3f7 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:50:22.163337 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1119/2242] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:50:22 2022] PUT /identity/v3/projects/2ded14adc3704a139c3e310030b1f5bc/users/bbd3041dc72e4db29a7eccb276e00de4/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 10:50:22.167178 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8e1e12-19b3-4272-9c25-a3a000109a4f admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:50:22.178363 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8e1e12-19b3-4272-9c25-a3a000109a4f admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:50:22.254525 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46ca4b4-2358-46da-bb07-2336a9179262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:50:22.255248 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ca4b4-2358-46da-bb07-2336a9179262 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:22.255672 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ca4b4-2358-46da-bb07-2336a9179262 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:22.256041 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ca4b4-2358-46da-bb07-2336a9179262 admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:22.256861 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46ca4b4-2358-46da-bb07-2336a9179262 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:50:22.258621 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46ca4b4-2358-46da-bb07-2336a9179262 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:50:22.270500 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1124/2243] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:50:22 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:50:22.275943 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d50f95-8c64-49b7-a565-630f7a78c3f7 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:50:22.289169 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d50f95-8c64-49b7-a565-630f7a78c3f7 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:50:22.294334 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833a45da-3ce1-43ae-ab61-cb634aac5847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:50:22.295133 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833a45da-3ce1-43ae-ab61-cb634aac5847 admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:22.295468 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833a45da-3ce1-43ae-ab61-cb634aac5847 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:22.295754 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833a45da-3ce1-43ae-ab61-cb634aac5847 admin admin] PATH_INFO: `/v3/projects/2ded14adc3704a139c3e310030b1f5bc/users/bbd3041dc72e4db29a7eccb276e00de4/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:22.297900 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833a45da-3ce1-43ae-ab61-cb634aac5847 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ded14adc3704a139c3e310030b1f5bc, user_id=bbd3041dc72e4db29a7eccb276e00de4, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:50:22.300607 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833a45da-3ce1-43ae-ab61-cb634aac5847 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:50:22.315322 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1120/2244] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:50:22 2022] PUT /identity/v3/projects/2ded14adc3704a139c3e310030b1f5bc/users/bbd3041dc72e4db29a7eccb276e00de4/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:50:27.399127 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46ca4b4-2358-46da-bb07-2336a9179262 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:50:27.399905 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-f46ca4b4-2358-46da-bb07-2336a9179262 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:50:27.419457 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46ca4b4-2358-46da-bb07-2336a9179262 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:50:27.591116 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58353fa4-d11d-48df-8936-05bb0194bfe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:50:27.593242 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58353fa4-d11d-48df-8936-05bb0194bfe4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:27.593774 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58353fa4-d11d-48df-8936-05bb0194bfe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:27.594444 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58353fa4-d11d-48df-8936-05bb0194bfe4 admin admin] PATH_INFO: `/v3/users/8ebd403c3d07406f82d0163b7bb0bd99` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:27.596073 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58353fa4-d11d-48df-8936-05bb0194bfe4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ebd403c3d07406f82d0163b7bb0bd99)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:50:27.599692 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58353fa4-d11d-48df-8936-05bb0194bfe4 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:50:27.652583 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:50:27.715163 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-58353fa4-d11d-48df-8936-05bb0194bfe4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ebd403c3d07406f82d0163b7bb0bd99'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:50:27.728687 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-58353fa4-d11d-48df-8936-05bb0194bfe4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ebd403c3d07406f82d0163b7bb0bd99'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:50:27.729787 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-58353fa4-d11d-48df-8936-05bb0194bfe4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ebd403c3d07406f82d0163b7bb0bd99'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:50:27.730617 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-58353fa4-d11d-48df-8936-05bb0194bfe4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ebd403c3d07406f82d0163b7bb0bd99'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:50:27.753775 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-58353fa4-d11d-48df-8936-05bb0194bfe4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ebd403c3d07406f82d0163b7bb0bd99'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:50:27.777683 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1125/2245] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:50:27 2022] DELETE /identity/v3/users/8ebd403c3d07406f82d0163b7bb0bd99 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:50:27.778716 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4ece74-3abc-4f8d-a2f0-78bf93a928fc None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:27.779979 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4ece74-3abc-4f8d-a2f0-78bf93a928fc None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:27.780315 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4ece74-3abc-4f8d-a2f0-78bf93a928fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:27.813333 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-8a4ece74-3abc-4f8d-a2f0-78bf93a928fc None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:50:27.814943 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-8a4ece74-3abc-4f8d-a2f0-78bf93a928fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:50:27.901038 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1121/2246] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:50:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:50:27.918893 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58353fa4-d11d-48df-8936-05bb0194bfe4 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:50:27.921113 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-58353fa4-d11d-48df-8936-05bb0194bfe4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:50:28.025315 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58353fa4-d11d-48df-8936-05bb0194bfe4 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:50:28.032928 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe4c7b0-ac42-41d7-80a9-b7aa5eb8ef26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:50:28.033689 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe4c7b0-ac42-41d7-80a9-b7aa5eb8ef26 service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:28.033979 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe4c7b0-ac42-41d7-80a9-b7aa5eb8ef26 service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:28.034254 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe4c7b0-ac42-41d7-80a9-b7aa5eb8ef26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:28.053314 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe4c7b0-ac42-41d7-80a9-b7aa5eb8ef26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:50:28.054913 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe4c7b0-ac42-41d7-80a9-b7aa5eb8ef26 service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:50:28.077281 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1126/2247] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:50:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:50:28.797170 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4ece74-3abc-4f8d-a2f0-78bf93a928fc None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:50:28.800373 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-8a4ece74-3abc-4f8d-a2f0-78bf93a928fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:50:28.818200 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4ece74-3abc-4f8d-a2f0-78bf93a928fc None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:50:28.833043 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3781b12-41fc-4bf5-bf3d-0580e9d5c2fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:50:28.834199 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3781b12-41fc-4bf5-bf3d-0580e9d5c2fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:28.834708 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3781b12-41fc-4bf5-bf3d-0580e9d5c2fc admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:28.835080 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3781b12-41fc-4bf5-bf3d-0580e9d5c2fc admin admin] PATH_INFO: `/v3/projects/52916d27e2e84d9c898c2084cdbdbb42` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:28.836521 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3781b12-41fc-4bf5-bf3d-0580e9d5c2fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=52916d27e2e84d9c898c2084cdbdbb42)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:50:28.839027 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3781b12-41fc-4bf5-bf3d-0580e9d5c2fc admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:50:28.894675 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28c9f99-bc6e-4fd9-b5d4-fdbb911c233c None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:28.895653 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28c9f99-bc6e-4fd9-b5d4-fdbb911c233c None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:28.896324 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28c9f99-bc6e-4fd9-b5d4-fdbb911c233c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:28.901425 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:50:28.956191 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-c28c9f99-bc6e-4fd9-b5d4-fdbb911c233c None None] MFA Rules not processed for user `bbd3041dc72e4db29a7eccb276e00de4`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:50:28.957546 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-c28c9f99-bc6e-4fd9-b5d4-fdbb911c233c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:50:28.991805 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-f3781b12-41fc-4bf5-bf3d-0580e9d5c2fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52916d27e2e84d9c898c2084cdbdbb42', 'request_id': 'req-f3781b12-41fc-4bf5-bf3d-0580e9d5c2fc'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:50:29.010411 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-f3781b12-41fc-4bf5-bf3d-0580e9d5c2fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52916d27e2e84d9c898c2084cdbdbb42', 'request_id': 'req-f3781b12-41fc-4bf5-bf3d-0580e9d5c2fc'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:50:29.024901 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-f3781b12-41fc-4bf5-bf3d-0580e9d5c2fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52916d27e2e84d9c898c2084cdbdbb42', 'request_id': 'req-f3781b12-41fc-4bf5-bf3d-0580e9d5c2fc'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:50:29.043896 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-f3781b12-41fc-4bf5-bf3d-0580e9d5c2fc admin admin] The token cache is being invalidate because project 52916d27e2e84d9c898c2084cdbdbb42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90653) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:50:29.044329 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-f3781b12-41fc-4bf5-bf3d-0580e9d5c2fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:50:29.045919 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-f3781b12-41fc-4bf5-bf3d-0580e9d5c2fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:50:29.047735 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1122/2248] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:50:28 2022] DELETE /identity/v3/projects/52916d27e2e84d9c898c2084cdbdbb42 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:50:29.051716 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3781b12-41fc-4bf5-bf3d-0580e9d5c2fc admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:50:29.053413 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-f3781b12-41fc-4bf5-bf3d-0580e9d5c2fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:50:29.053555 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1127/2249] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:50:28 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:50:29.117727 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28c9f99-bc6e-4fd9-b5d4-fdbb911c233c None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:50:29.119941 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-c28c9f99-bc6e-4fd9-b5d4-fdbb911c233c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:50:29.141352 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3781b12-41fc-4bf5-bf3d-0580e9d5c2fc admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:50:29.153291 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f923b54a-7db0-478c-bcc8-365ebbba6a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:50:29.154571 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f923b54a-7db0-478c-bcc8-365ebbba6a8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:29.155597 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f923b54a-7db0-478c-bcc8-365ebbba6a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:29.155875 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f923b54a-7db0-478c-bcc8-365ebbba6a8b admin admin] PATH_INFO: `/v3/users/84b8e4ab226747cb9bba2a31a54c0d81` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:29.157119 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f923b54a-7db0-478c-bcc8-365ebbba6a8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=84b8e4ab226747cb9bba2a31a54c0d81)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:50:29.159081 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f923b54a-7db0-478c-bcc8-365ebbba6a8b admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:50:29.181375 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:50:29.223097 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-f923b54a-7db0-478c-bcc8-365ebbba6a8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84b8e4ab226747cb9bba2a31a54c0d81'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:50:29.235282 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-f923b54a-7db0-478c-bcc8-365ebbba6a8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84b8e4ab226747cb9bba2a31a54c0d81'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:50:29.236054 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-f923b54a-7db0-478c-bcc8-365ebbba6a8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84b8e4ab226747cb9bba2a31a54c0d81'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:50:29.237266 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-f923b54a-7db0-478c-bcc8-365ebbba6a8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84b8e4ab226747cb9bba2a31a54c0d81'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:50:29.239671 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28c9f99-bc6e-4fd9-b5d4-fdbb911c233c None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:50:29.247680 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-f923b54a-7db0-478c-bcc8-365ebbba6a8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84b8e4ab226747cb9bba2a31a54c0d81'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:50:29.276124 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1123/2250] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:50:29 2022] DELETE /identity/v3/users/84b8e4ab226747cb9bba2a31a54c0d81 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 10:50:29.346466 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292a4fe8-1e52-42a6-815f-3e7aebf26e46 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bd5356829b6454993bab9bad2f79fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:50:29.347589 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a4fe8-1e52-42a6-815f-3e7aebf26e46 service nova] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:29.348319 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a4fe8-1e52-42a6-815f-3e7aebf26e46 service nova] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:29.348427 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a4fe8-1e52-42a6-815f-3e7aebf26e46 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:29.350905 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-292a4fe8-1e52-42a6-815f-3e7aebf26e46 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:50:29.429225 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292a4fe8-1e52-42a6-815f-3e7aebf26e46 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:50:29.430952 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292a4fe8-1e52-42a6-815f-3e7aebf26e46 service nova] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:50:29.443792 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1128/2251] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:50:29 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 10:50:29.601178 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f923b54a-7db0-478c-bcc8-365ebbba6a8b admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:50:29.603283 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-f923b54a-7db0-478c-bcc8-365ebbba6a8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:50:29.625342 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:50:29.692465 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f923b54a-7db0-478c-bcc8-365ebbba6a8b admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:50:29.697599 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7d81c9-4540-4d11-8353-11ad9a0ee414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:50:29.698244 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7d81c9-4540-4d11-8353-11ad9a0ee414 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:29.698528 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7d81c9-4540-4d11-8353-11ad9a0ee414 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:29.698803 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7d81c9-4540-4d11-8353-11ad9a0ee414 admin admin] PATH_INFO: `/v3/projects/df6b2c180247434f87eb7914685682c0` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:29.700037 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7d81c9-4540-4d11-8353-11ad9a0ee414 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df6b2c180247434f87eb7914685682c0)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:50:29.702579 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7d81c9-4540-4d11-8353-11ad9a0ee414 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:50:29.764155 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-bb7d81c9-4540-4d11-8353-11ad9a0ee414 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df6b2c180247434f87eb7914685682c0', 'request_id': 'req-bb7d81c9-4540-4d11-8353-11ad9a0ee414'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:50:29.771213 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-bb7d81c9-4540-4d11-8353-11ad9a0ee414 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df6b2c180247434f87eb7914685682c0', 'request_id': 'req-bb7d81c9-4540-4d11-8353-11ad9a0ee414'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:50:29.784373 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-bb7d81c9-4540-4d11-8353-11ad9a0ee414 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df6b2c180247434f87eb7914685682c0', 'request_id': 'req-bb7d81c9-4540-4d11-8353-11ad9a0ee414'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:50:29.797724 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-bb7d81c9-4540-4d11-8353-11ad9a0ee414 admin admin] The token cache is being invalidate because project df6b2c180247434f87eb7914685682c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90653) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:50:29.798384 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-bb7d81c9-4540-4d11-8353-11ad9a0ee414 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:50:29.798851 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-bb7d81c9-4540-4d11-8353-11ad9a0ee414 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:50:29.800401 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1124/2252] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:50:29 2022] DELETE /identity/v3/projects/df6b2c180247434f87eb7914685682c0 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:50:29.805352 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02657903-ab83-482c-8995-eae035a4e3c6 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:29.805643 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02657903-ab83-482c-8995-eae035a4e3c6 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:29.805919 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02657903-ab83-482c-8995-eae035a4e3c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:29.845679 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-02657903-ab83-482c-8995-eae035a4e3c6 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:50:29.847337 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-02657903-ab83-482c-8995-eae035a4e3c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:50:29.936154 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1129/2253] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:50:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:50:29.943807 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f81600-ea67-49cb-adf9-5b62bc29a8c0 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:29.944093 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f81600-ea67-49cb-adf9-5b62bc29a8c0 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:29.944371 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f81600-ea67-49cb-adf9-5b62bc29a8c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:29.966148 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-72f81600-ea67-49cb-adf9-5b62bc29a8c0 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:50:29.968073 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-72f81600-ea67-49cb-adf9-5b62bc29a8c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:50:29.978025 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1125/2254] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:50:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 10:50:29.983117 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02657903-ab83-482c-8995-eae035a4e3c6 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:50:29.994183 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02657903-ab83-482c-8995-eae035a4e3c6 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:50:30.000252 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1378cf50-f30a-40fe-a302-f442d07c4e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:50:30.001387 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1378cf50-f30a-40fe-a302-f442d07c4e81 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:30.001387 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1378cf50-f30a-40fe-a302-f442d07c4e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:30.001626 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1378cf50-f30a-40fe-a302-f442d07c4e81 admin admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:30.002332 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1378cf50-f30a-40fe-a302-f442d07c4e81 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:50:30.002708 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1378cf50-f30a-40fe-a302-f442d07c4e81 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:50:30.004598 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1378cf50-f30a-40fe-a302-f442d07c4e81 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:50:30.007360 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:50:30.019390 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1130/2255] 10.222.0.15 () {58 vars in 1153 bytes} [Sun Sep 11 10:50:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:50:30.023741 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f81600-ea67-49cb-adf9-5b62bc29a8c0 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:50:30.035198 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f81600-ea67-49cb-adf9-5b62bc29a8c0 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:50:30.040070 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f60052-5962-4883-8afc-d50e455c12ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:50:30.040684 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f60052-5962-4883-8afc-d50e455c12ee admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:30.040975 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f60052-5962-4883-8afc-d50e455c12ee admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:30.041248 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f60052-5962-4883-8afc-d50e455c12ee admin admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:30.041883 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f60052-5962-4883-8afc-d50e455c12ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:50:30.043958 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f60052-5962-4883-8afc-d50e455c12ee admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:50:30.078572 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1126/2256] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:50:30 2022] POST /identity/v3/projects => generated 363 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:50:30.082912 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1378cf50-f30a-40fe-a302-f442d07c4e81 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:50:30.094590 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1378cf50-f30a-40fe-a302-f442d07c4e81 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:50:30.166877 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729e05dc-4b01-4496-9c8e-ff4f398485be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:50:30.167530 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729e05dc-4b01-4496-9c8e-ff4f398485be admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:30.167817 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729e05dc-4b01-4496-9c8e-ff4f398485be admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:30.168235 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729e05dc-4b01-4496-9c8e-ff4f398485be admin admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:30.169028 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729e05dc-4b01-4496-9c8e-ff4f398485be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:50:30.172072 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729e05dc-4b01-4496-9c8e-ff4f398485be admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:50:30.197622 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1131/2257] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:50:30 2022] POST /identity/v3/users => generated 282 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:50:30.201494 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f60052-5962-4883-8afc-d50e455c12ee admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:50:30.213590 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f60052-5962-4883-8afc-d50e455c12ee admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:50:30.220143 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26a12ee-1be2-4107-afbf-8f2cd7ffe4b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:50:30.220964 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a12ee-1be2-4107-afbf-8f2cd7ffe4b0 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:30.221871 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a12ee-1be2-4107-afbf-8f2cd7ffe4b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:30.221871 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a12ee-1be2-4107-afbf-8f2cd7ffe4b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:30.222316 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26a12ee-1be2-4107-afbf-8f2cd7ffe4b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:50:30.224021 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26a12ee-1be2-4107-afbf-8f2cd7ffe4b0 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:50:30.238813 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1127/2258] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:50:30 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:50:30.241246 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729e05dc-4b01-4496-9c8e-ff4f398485be admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:50:30.257427 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729e05dc-4b01-4496-9c8e-ff4f398485be admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:50:30.265637 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e38959e-9f24-4fa7-b83f-f6f5545dd8a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:50:30.266796 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e38959e-9f24-4fa7-b83f-f6f5545dd8a5 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:30.266796 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e38959e-9f24-4fa7-b83f-f6f5545dd8a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:30.267097 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e38959e-9f24-4fa7-b83f-f6f5545dd8a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:30.267630 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e38959e-9f24-4fa7-b83f-f6f5545dd8a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:50:30.269679 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e38959e-9f24-4fa7-b83f-f6f5545dd8a5 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:50:30.291044 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1132/2259] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:50:30 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:50:30.293555 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26a12ee-1be2-4107-afbf-8f2cd7ffe4b0 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:50:30.325950 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26a12ee-1be2-4107-afbf-8f2cd7ffe4b0 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:50:30.343673 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5ab812-a3bd-416d-92c1-f6cddee34944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:50:30.344708 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5ab812-a3bd-416d-92c1-f6cddee34944 admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:30.345141 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5ab812-a3bd-416d-92c1-f6cddee34944 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:30.345574 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5ab812-a3bd-416d-92c1-f6cddee34944 admin admin] PATH_INFO: `/v3/projects/8dfb8c9ae71745b28927a3db6faf00d0/users/92c1835d7f814f9aafdf042868647561/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:30.374305 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5ab812-a3bd-416d-92c1-f6cddee34944 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dfb8c9ae71745b28927a3db6faf00d0, user_id=92c1835d7f814f9aafdf042868647561, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:50:30.377014 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5ab812-a3bd-416d-92c1-f6cddee34944 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:50:30.399876 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1128/2260] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:50:30 2022] PUT /identity/v3/projects/8dfb8c9ae71745b28927a3db6faf00d0/users/92c1835d7f814f9aafdf042868647561/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 10:50:30.405264 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d6782d-0bde-45a2-8eab-3d193e1c0e61 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:30.405568 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d6782d-0bde-45a2-8eab-3d193e1c0e61 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:30.405878 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d6782d-0bde-45a2-8eab-3d193e1c0e61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:30.450198 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-50d6782d-0bde-45a2-8eab-3d193e1c0e61 None None] MFA Rules not processed for user `92c1835d7f814f9aafdf042868647561`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:50:30.453841 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-50d6782d-0bde-45a2-8eab-3d193e1c0e61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:50:30.547499 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1133/2261] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:50:30 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:50:30.551814 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5ab812-a3bd-416d-92c1-f6cddee34944 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:50:30.615492 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:50:30.650079 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5ab812-a3bd-416d-92c1-f6cddee34944 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:50:30.658910 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d773c7a9-2911-43f2-a247-53fc632c7851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:50:30.659899 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d773c7a9-2911-43f2-a247-53fc632c7851 admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:30.660338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d773c7a9-2911-43f2-a247-53fc632c7851 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:30.660897 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d773c7a9-2911-43f2-a247-53fc632c7851 admin admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:30.662065 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d773c7a9-2911-43f2-a247-53fc632c7851 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:50:30.664426 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d773c7a9-2911-43f2-a247-53fc632c7851 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:50:30.707595 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1129/2262] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:50:30 2022] POST /identity/v3/projects => generated 363 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:50:30.711318 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d6782d-0bde-45a2-8eab-3d193e1c0e61 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:50:30.727727 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d6782d-0bde-45a2-8eab-3d193e1c0e61 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:50:30.811962 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f1e0f5-b174-4b87-b0e8-0a3546067ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:50:30.812722 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f1e0f5-b174-4b87-b0e8-0a3546067ce6 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:30.812992 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f1e0f5-b174-4b87-b0e8-0a3546067ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:30.813269 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f1e0f5-b174-4b87-b0e8-0a3546067ce6 admin admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:30.813970 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f1e0f5-b174-4b87-b0e8-0a3546067ce6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:50:30.815973 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f1e0f5-b174-4b87-b0e8-0a3546067ce6 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:50:30.827763 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:50:30.843542 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1134/2263] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:50:30 2022] POST /identity/v3/users => generated 288 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:50:30.846424 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d773c7a9-2911-43f2-a247-53fc632c7851 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:50:30.863974 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d773c7a9-2911-43f2-a247-53fc632c7851 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:50:30.870642 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859fec87-a07f-4e7d-8b3f-7b7e848c83a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:50:30.871358 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859fec87-a07f-4e7d-8b3f-7b7e848c83a6 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:30.871772 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859fec87-a07f-4e7d-8b3f-7b7e848c83a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:30.872139 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859fec87-a07f-4e7d-8b3f-7b7e848c83a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:30.874094 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859fec87-a07f-4e7d-8b3f-7b7e848c83a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:50:30.875906 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859fec87-a07f-4e7d-8b3f-7b7e848c83a6 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:50:30.896353 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1130/2264] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:50:30 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:50:30.897570 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f1e0f5-b174-4b87-b0e8-0a3546067ce6 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:50:30.913484 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f1e0f5-b174-4b87-b0e8-0a3546067ce6 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:50:30.919954 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3a9868-57d0-4c9f-97a1-5c98a3bd1cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:50:30.921730 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3a9868-57d0-4c9f-97a1-5c98a3bd1cd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:30.921730 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3a9868-57d0-4c9f-97a1-5c98a3bd1cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:30.921730 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3a9868-57d0-4c9f-97a1-5c98a3bd1cd8 admin admin] PATH_INFO: `/v3/projects/55c4dbe5d204439390c914b1b746e0ee/users/df0919ca27e74e728356ffcaeb3b9402/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:30.937596 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3a9868-57d0-4c9f-97a1-5c98a3bd1cd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55c4dbe5d204439390c914b1b746e0ee, user_id=df0919ca27e74e728356ffcaeb3b9402, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:50:30.940343 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3a9868-57d0-4c9f-97a1-5c98a3bd1cd8 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:50:30.959475 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1135/2265] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:50:30 2022] PUT /identity/v3/projects/55c4dbe5d204439390c914b1b746e0ee/users/df0919ca27e74e728356ffcaeb3b9402/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:50:30.961776 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859fec87-a07f-4e7d-8b3f-7b7e848c83a6 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:50:30.974024 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859fec87-a07f-4e7d-8b3f-7b7e848c83a6 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:50:31.038396 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0574d8-9d1e-4074-9ae7-855d2308e8ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:50:31.039355 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0574d8-9d1e-4074-9ae7-855d2308e8ca admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:31.040819 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0574d8-9d1e-4074-9ae7-855d2308e8ca admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:31.040819 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0574d8-9d1e-4074-9ae7-855d2308e8ca admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:31.041529 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0574d8-9d1e-4074-9ae7-855d2308e8ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:50:31.043272 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0574d8-9d1e-4074-9ae7-855d2308e8ca admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:50:31.056838 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1131/2266] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:50:30 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:50:31.062083 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3a9868-57d0-4c9f-97a1-5c98a3bd1cd8 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:50:31.073203 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3a9868-57d0-4c9f-97a1-5c98a3bd1cd8 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:50:31.079786 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144f9f3b-fcf8-4b8d-a0ac-76c28f73091e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:50:31.080782 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144f9f3b-fcf8-4b8d-a0ac-76c28f73091e admin admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:31.081206 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144f9f3b-fcf8-4b8d-a0ac-76c28f73091e admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:31.081631 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144f9f3b-fcf8-4b8d-a0ac-76c28f73091e admin admin] PATH_INFO: `/v3/projects/55c4dbe5d204439390c914b1b746e0ee/users/df0919ca27e74e728356ffcaeb3b9402/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:31.083946 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144f9f3b-fcf8-4b8d-a0ac-76c28f73091e admin admin] RBAC: Authorizing `identity:create_grant(project_id=55c4dbe5d204439390c914b1b746e0ee, user_id=df0919ca27e74e728356ffcaeb3b9402, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:50:31.086763 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144f9f3b-fcf8-4b8d-a0ac-76c28f73091e admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:50:31.099391 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1136/2267] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:50:31 2022] PUT /identity/v3/projects/55c4dbe5d204439390c914b1b746e0ee/users/df0919ca27e74e728356ffcaeb3b9402/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 10:50:31.113398 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129ae490-0d01-4795-8966-300a500c83ff None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:31.113947 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129ae490-0d01-4795-8966-300a500c83ff None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:31.114994 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129ae490-0d01-4795-8966-300a500c83ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:31.189333 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-129ae490-0d01-4795-8966-300a500c83ff None None] MFA Rules not processed for user `df0919ca27e74e728356ffcaeb3b9402`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:50:31.190614 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-129ae490-0d01-4795-8966-300a500c83ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:50:31.381944 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1132/2268] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:50:31 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 10:50:31.393490 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144f9f3b-fcf8-4b8d-a0ac-76c28f73091e admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:50:31.395245 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-144f9f3b-fcf8-4b8d-a0ac-76c28f73091e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:50:31.531447 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144f9f3b-fcf8-4b8d-a0ac-76c28f73091e admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:50:31.541232 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6377a06f-aedf-4399-958b-fcacdf44ade0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:50:31.541885 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6377a06f-aedf-4399-958b-fcacdf44ade0 service cinder] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:31.542171 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6377a06f-aedf-4399-958b-fcacdf44ade0 service cinder] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:31.542447 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6377a06f-aedf-4399-958b-fcacdf44ade0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:31.564155 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6377a06f-aedf-4399-958b-fcacdf44ade0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:50:31.565822 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6377a06f-aedf-4399-958b-fcacdf44ade0 service cinder] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:50:31.595296 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1137/2269] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:50:31 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:50:31.745355 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129ae490-0d01-4795-8966-300a500c83ff None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:50:31.761559 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129ae490-0d01-4795-8966-300a500c83ff None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:50:31.769280 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74227edb-89f3-4cb3-9d48-8d75954ff224 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:50:31.769915 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74227edb-89f3-4cb3-9d48-8d75954ff224 service cinder] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:31.770202 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74227edb-89f3-4cb3-9d48-8d75954ff224 service cinder] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:31.770478 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74227edb-89f3-4cb3-9d48-8d75954ff224 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:31.866281 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:50:31.918809 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74227edb-89f3-4cb3-9d48-8d75954ff224 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:50:31.921032 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74227edb-89f3-4cb3-9d48-8d75954ff224 service cinder] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:50:31.937147 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1133/2270] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:50:31 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:50:41.584061 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:INSERT=3,keystone:SELECT=43 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:50:41.929079 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=12 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:50:50.053208 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9799ada9-a097-406b-b307-abf260b2695b None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:50.053674 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9799ada9-a097-406b-b307-abf260b2695b None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:50.053905 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9799ada9-a097-406b-b307-abf260b2695b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:50.079457 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-9799ada9-a097-406b-b307-abf260b2695b None None] MFA Rules not processed for user `bb44e7aef0c7436d93a7233307b5f095`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:50:50.080670 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-9799ada9-a097-406b-b307-abf260b2695b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:50:50.139973 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1138/2271] 10.222.0.15 () {58 vars in 1009 bytes} [Sun Sep 11 10:50:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:50:50.185283 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74227edb-89f3-4cb3-9d48-8d75954ff224 service cinder] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:50:50.187519 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-74227edb-89f3-4cb3-9d48-8d75954ff224 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:50:50.272128 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74227edb-89f3-4cb3-9d48-8d75954ff224 service cinder] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:50:50.283188 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3518074-944b-4757-b36a-de3041116d75 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7ca566e66a45b29e3d02f47bf84981', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:50:50.283188 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3518074-944b-4757-b36a-de3041116d75 service swift] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:50.283188 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3518074-944b-4757-b36a-de3041116d75 service swift] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:50.283188 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3518074-944b-4757-b36a-de3041116d75 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:50.291771 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3518074-944b-4757-b36a-de3041116d75 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:50:50.293364 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3518074-944b-4757-b36a-de3041116d75 service swift] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:50:50.302224 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1134/2272] 10.222.0.15 () {60 vars in 1384 bytes} [Sun Sep 11 10:50:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 10:50:54.202873 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9799ada9-a097-406b-b307-abf260b2695b None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:50:54.271773 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9799ada9-a097-406b-b307-abf260b2695b None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:50:54.276755 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08a132a-dd18-43b5-8b49-c32dce916ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:50:54.277390 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08a132a-dd18-43b5-8b49-c32dce916ab9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:54.277675 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08a132a-dd18-43b5-8b49-c32dce916ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:54.278400 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08a132a-dd18-43b5-8b49-c32dce916ab9 admin admin] PATH_INFO: `/v3/users/92c1835d7f814f9aafdf042868647561` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:54.279227 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08a132a-dd18-43b5-8b49-c32dce916ab9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=92c1835d7f814f9aafdf042868647561)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:50:54.281829 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08a132a-dd18-43b5-8b49-c32dce916ab9 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:50:54.343059 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-d08a132a-dd18-43b5-8b49-c32dce916ab9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92c1835d7f814f9aafdf042868647561'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:50:54.354262 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-d08a132a-dd18-43b5-8b49-c32dce916ab9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92c1835d7f814f9aafdf042868647561'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:50:54.354981 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-d08a132a-dd18-43b5-8b49-c32dce916ab9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92c1835d7f814f9aafdf042868647561'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:50:54.356138 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-d08a132a-dd18-43b5-8b49-c32dce916ab9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92c1835d7f814f9aafdf042868647561'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:50:54.372852 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-d08a132a-dd18-43b5-8b49-c32dce916ab9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92c1835d7f814f9aafdf042868647561'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:50:54.388057 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1139/2273] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:50:54 2022] DELETE /identity/v3/users/92c1835d7f814f9aafdf042868647561 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:50:54.392809 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec499b6-861e-4df6-af45-2a8c69544e0a None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:54.393351 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec499b6-861e-4df6-af45-2a8c69544e0a None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:54.393492 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec499b6-861e-4df6-af45-2a8c69544e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:54.428409 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-fec499b6-861e-4df6-af45-2a8c69544e0a None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:50:54.430366 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-fec499b6-861e-4df6-af45-2a8c69544e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:50:54.528111 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1135/2274] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:50:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:50:54.535287 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08a132a-dd18-43b5-8b49-c32dce916ab9 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:50:54.537172 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-d08a132a-dd18-43b5-8b49-c32dce916ab9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:50:54.584705 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:50:54.622378 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08a132a-dd18-43b5-8b49-c32dce916ab9 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:50:54.626532 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f78e84c-f5ec-400c-ba74-374fda58e689 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:50:54.627296 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f78e84c-f5ec-400c-ba74-374fda58e689 service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:54.627717 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f78e84c-f5ec-400c-ba74-374fda58e689 service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:54.628099 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f78e84c-f5ec-400c-ba74-374fda58e689 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:54.639375 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f78e84c-f5ec-400c-ba74-374fda58e689 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:50:54.641069 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f78e84c-f5ec-400c-ba74-374fda58e689 service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:50:54.651969 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1140/2275] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:50:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:50:55.040577 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec499b6-861e-4df6-af45-2a8c69544e0a None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:50:55.042755 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-fec499b6-861e-4df6-af45-2a8c69544e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:50:55.065603 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec499b6-861e-4df6-af45-2a8c69544e0a None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:50:55.070714 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37933a9-3133-423a-9084-298d8e132c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:50:55.071464 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37933a9-3133-423a-9084-298d8e132c13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:55.071778 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37933a9-3133-423a-9084-298d8e132c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:55.072085 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37933a9-3133-423a-9084-298d8e132c13 admin admin] PATH_INFO: `/v3/projects/8dfb8c9ae71745b28927a3db6faf00d0` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:55.073315 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37933a9-3133-423a-9084-298d8e132c13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8dfb8c9ae71745b28927a3db6faf00d0)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:50:55.095714 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37933a9-3133-423a-9084-298d8e132c13 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:50:55.159047 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-b37933a9-3133-423a-9084-298d8e132c13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8dfb8c9ae71745b28927a3db6faf00d0', 'request_id': 'req-b37933a9-3133-423a-9084-298d8e132c13'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:50:55.166867 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-b37933a9-3133-423a-9084-298d8e132c13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8dfb8c9ae71745b28927a3db6faf00d0', 'request_id': 'req-b37933a9-3133-423a-9084-298d8e132c13'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:50:55.177544 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-b37933a9-3133-423a-9084-298d8e132c13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8dfb8c9ae71745b28927a3db6faf00d0', 'request_id': 'req-b37933a9-3133-423a-9084-298d8e132c13'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:50:55.190429 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-b37933a9-3133-423a-9084-298d8e132c13 admin admin] The token cache is being invalidate because project 8dfb8c9ae71745b28927a3db6faf00d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90653) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:50:55.190721 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-b37933a9-3133-423a-9084-298d8e132c13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:50:55.191378 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-b37933a9-3133-423a-9084-298d8e132c13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:50:55.196461 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1136/2276] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:50:55 2022] DELETE /identity/v3/projects/8dfb8c9ae71745b28927a3db6faf00d0 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:50:55.198265 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f78e84c-f5ec-400c-ba74-374fda58e689 service neutron] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:50:55.210267 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f78e84c-f5ec-400c-ba74-374fda58e689 service neutron] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:50:55.327632 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2337d7-6b13-4289-97d3-aeda2c71b423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:50:55.328736 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2337d7-6b13-4289-97d3-aeda2c71b423 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:55.329029 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2337d7-6b13-4289-97d3-aeda2c71b423 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:55.329309 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2337d7-6b13-4289-97d3-aeda2c71b423 admin admin] PATH_INFO: `/v3/users/df0919ca27e74e728356ffcaeb3b9402` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:55.330695 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2337d7-6b13-4289-97d3-aeda2c71b423 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df0919ca27e74e728356ffcaeb3b9402)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:50:55.337857 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2337d7-6b13-4289-97d3-aeda2c71b423 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:50:55.408568 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-9c2337d7-6b13-4289-97d3-aeda2c71b423 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df0919ca27e74e728356ffcaeb3b9402'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:50:55.416461 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-9c2337d7-6b13-4289-97d3-aeda2c71b423 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df0919ca27e74e728356ffcaeb3b9402'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:50:55.417079 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-9c2337d7-6b13-4289-97d3-aeda2c71b423 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df0919ca27e74e728356ffcaeb3b9402'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:50:55.417734 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-9c2337d7-6b13-4289-97d3-aeda2c71b423 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df0919ca27e74e728356ffcaeb3b9402'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:50:55.433384 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-9c2337d7-6b13-4289-97d3-aeda2c71b423 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df0919ca27e74e728356ffcaeb3b9402'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:50:55.462810 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1141/2277] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:50:55 2022] DELETE /identity/v3/users/df0919ca27e74e728356ffcaeb3b9402 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:50:55.752666 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37933a9-3133-423a-9084-298d8e132c13 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:50:55.754225 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-b37933a9-3133-423a-9084-298d8e132c13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:50:55.768543 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37933a9-3133-423a-9084-298d8e132c13 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:50:55.780246 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:50:55.838034 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08f2e92-308b-4237-abb5-4e8e9ba7adf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:50:55.839089 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08f2e92-308b-4237-abb5-4e8e9ba7adf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:55.839670 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08f2e92-308b-4237-abb5-4e8e9ba7adf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:55.840226 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08f2e92-308b-4237-abb5-4e8e9ba7adf2 admin admin] PATH_INFO: `/v3/projects/55c4dbe5d204439390c914b1b746e0ee` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:55.841877 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08f2e92-308b-4237-abb5-4e8e9ba7adf2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55c4dbe5d204439390c914b1b746e0ee)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:50:55.844954 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08f2e92-308b-4237-abb5-4e8e9ba7adf2 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:50:55.901275 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-c08f2e92-308b-4237-abb5-4e8e9ba7adf2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55c4dbe5d204439390c914b1b746e0ee', 'request_id': 'req-c08f2e92-308b-4237-abb5-4e8e9ba7adf2'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:50:55.908677 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-c08f2e92-308b-4237-abb5-4e8e9ba7adf2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55c4dbe5d204439390c914b1b746e0ee', 'request_id': 'req-c08f2e92-308b-4237-abb5-4e8e9ba7adf2'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:50:55.919324 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-c08f2e92-308b-4237-abb5-4e8e9ba7adf2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55c4dbe5d204439390c914b1b746e0ee', 'request_id': 'req-c08f2e92-308b-4237-abb5-4e8e9ba7adf2'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:50:55.933538 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-c08f2e92-308b-4237-abb5-4e8e9ba7adf2 admin admin] The token cache is being invalidate because project 55c4dbe5d204439390c914b1b746e0ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90653) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:50:55.933856 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-c08f2e92-308b-4237-abb5-4e8e9ba7adf2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:50:55.934958 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-c08f2e92-308b-4237-abb5-4e8e9ba7adf2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:50:55.936885 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1137/2278] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:50:55 2022] DELETE /identity/v3/projects/55c4dbe5d204439390c914b1b746e0ee => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:50:55.942062 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b81acf-9e84-414c-b4cd-7e208801ebc4 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:55.942562 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b81acf-9e84-414c-b4cd-7e208801ebc4 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:55.942690 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b81acf-9e84-414c-b4cd-7e208801ebc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:55.966086 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-60b81acf-9e84-414c-b4cd-7e208801ebc4 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:50:55.968051 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-60b81acf-9e84-414c-b4cd-7e208801ebc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:50:56.020879 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:50:56.051545 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1142/2279] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:50:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:50:56.058828 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3288fc3-83e9-4566-be4b-94350557aedc None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:56.059187 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3288fc3-83e9-4566-be4b-94350557aedc None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:56.059564 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3288fc3-83e9-4566-be4b-94350557aedc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:56.084225 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-d3288fc3-83e9-4566-be4b-94350557aedc None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:50:56.085963 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-d3288fc3-83e9-4566-be4b-94350557aedc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:50:56.095494 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1138/2280] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:50:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:50:56.100251 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b81acf-9e84-414c-b4cd-7e208801ebc4 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:50:56.118708 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b81acf-9e84-414c-b4cd-7e208801ebc4 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:50:56.127392 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3512106-6d05-4df5-9c57-c48ac99c147f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:50:56.128574 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3512106-6d05-4df5-9c57-c48ac99c147f admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:56.129155 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3512106-6d05-4df5-9c57-c48ac99c147f admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:56.129732 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3512106-6d05-4df5-9c57-c48ac99c147f admin admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:56.130695 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3512106-6d05-4df5-9c57-c48ac99c147f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:50:56.131557 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3512106-6d05-4df5-9c57-c48ac99c147f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:50:56.133893 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3512106-6d05-4df5-9c57-c48ac99c147f admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:50:56.155166 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1143/2281] 10.222.0.15 () {58 vars in 1153 bytes} [Sun Sep 11 10:50:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:50:56.156748 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3288fc3-83e9-4566-be4b-94350557aedc None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:50:56.168293 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3288fc3-83e9-4566-be4b-94350557aedc None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:50:56.176544 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104edccd-a87c-4188-9231-9ae04e09bed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:50:56.177864 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104edccd-a87c-4188-9231-9ae04e09bed0 admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:56.178490 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104edccd-a87c-4188-9231-9ae04e09bed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:56.178986 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104edccd-a87c-4188-9231-9ae04e09bed0 admin admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:56.180280 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104edccd-a87c-4188-9231-9ae04e09bed0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:50:56.182625 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104edccd-a87c-4188-9231-9ae04e09bed0 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:50:56.228375 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3512106-6d05-4df5-9c57-c48ac99c147f admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:50:56.240253 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1139/2282] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:50:56 2022] POST /identity/v3/projects => generated 367 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:50:56.249858 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3512106-6d05-4df5-9c57-c48ac99c147f admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:50:56.349582 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8a62ce-99d0-4945-97be-aac30952bf40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:50:56.350369 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8a62ce-99d0-4945-97be-aac30952bf40 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:56.350788 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8a62ce-99d0-4945-97be-aac30952bf40 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:56.351212 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8a62ce-99d0-4945-97be-aac30952bf40 admin admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:56.352035 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8a62ce-99d0-4945-97be-aac30952bf40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:50:56.354203 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8a62ce-99d0-4945-97be-aac30952bf40 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:50:56.381426 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1144/2283] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:50:56 2022] POST /identity/v3/users => generated 284 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:50:56.386010 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104edccd-a87c-4188-9231-9ae04e09bed0 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:50:56.398144 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104edccd-a87c-4188-9231-9ae04e09bed0 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:50:56.404084 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b76418d-46c3-4253-a0d7-fea785f05e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:50:56.405474 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b76418d-46c3-4253-a0d7-fea785f05e1d admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:56.405718 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b76418d-46c3-4253-a0d7-fea785f05e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:56.406003 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b76418d-46c3-4253-a0d7-fea785f05e1d admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:56.406658 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b76418d-46c3-4253-a0d7-fea785f05e1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:50:56.408410 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b76418d-46c3-4253-a0d7-fea785f05e1d admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:50:56.423764 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1140/2284] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:50:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:50:56.424919 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8a62ce-99d0-4945-97be-aac30952bf40 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:50:56.436937 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8a62ce-99d0-4945-97be-aac30952bf40 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:50:56.445543 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157bd8ef-f92d-45ca-9e3d-563239a226a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:50:56.446259 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157bd8ef-f92d-45ca-9e3d-563239a226a2 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:56.446663 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157bd8ef-f92d-45ca-9e3d-563239a226a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:56.446964 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157bd8ef-f92d-45ca-9e3d-563239a226a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:56.447724 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157bd8ef-f92d-45ca-9e3d-563239a226a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:50:56.449546 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157bd8ef-f92d-45ca-9e3d-563239a226a2 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:50:56.462258 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1145/2285] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:50:56 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 10:50:56.466367 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b76418d-46c3-4253-a0d7-fea785f05e1d admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:50:56.477479 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b76418d-46c3-4253-a0d7-fea785f05e1d admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:50:56.484456 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348e85dc-960a-4c6b-b878-b6870d0fa7eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:50:56.485276 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348e85dc-960a-4c6b-b878-b6870d0fa7eb admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:56.485695 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348e85dc-960a-4c6b-b878-b6870d0fa7eb admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:56.486056 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348e85dc-960a-4c6b-b878-b6870d0fa7eb admin admin] PATH_INFO: `/v3/projects/b29a24009b4d47829d7f9acb064230c1/users/97f5e57b7b304f78a84ce2305645eb30/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:56.504619 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348e85dc-960a-4c6b-b878-b6870d0fa7eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b29a24009b4d47829d7f9acb064230c1, user_id=97f5e57b7b304f78a84ce2305645eb30, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:50:56.512096 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348e85dc-960a-4c6b-b878-b6870d0fa7eb admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:50:56.531486 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1141/2286] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:50:56 2022] PUT /identity/v3/projects/b29a24009b4d47829d7f9acb064230c1/users/97f5e57b7b304f78a84ce2305645eb30/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:50:56.540609 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a367ad-7bc5-4da7-89bf-bf586a86b39f None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:56.541012 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a367ad-7bc5-4da7-89bf-bf586a86b39f None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:56.541292 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a367ad-7bc5-4da7-89bf-bf586a86b39f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:56.575147 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-16a367ad-7bc5-4da7-89bf-bf586a86b39f None None] MFA Rules not processed for user `97f5e57b7b304f78a84ce2305645eb30`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:50:56.577408 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-16a367ad-7bc5-4da7-89bf-bf586a86b39f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:50:56.651072 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1146/2287] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:50:56 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:50:56.656387 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348e85dc-960a-4c6b-b878-b6870d0fa7eb admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:50:56.706351 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:50:56.731394 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348e85dc-960a-4c6b-b878-b6870d0fa7eb admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:50:56.737679 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3382a898-3efe-4533-afb4-874cde8d0609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:50:56.738312 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3382a898-3efe-4533-afb4-874cde8d0609 admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:56.738610 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3382a898-3efe-4533-afb4-874cde8d0609 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:56.738893 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3382a898-3efe-4533-afb4-874cde8d0609 admin admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:56.739721 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3382a898-3efe-4533-afb4-874cde8d0609 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:50:56.741757 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3382a898-3efe-4533-afb4-874cde8d0609 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:50:56.777706 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1142/2288] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:50:56 2022] POST /identity/v3/projects => generated 367 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:50:56.779938 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a367ad-7bc5-4da7-89bf-bf586a86b39f None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:50:56.792369 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a367ad-7bc5-4da7-89bf-bf586a86b39f None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:50:56.844103 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:50:56.867369 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fc8791-ea70-4fbc-ba60-ed46327e75c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:50:56.868291 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc8791-ea70-4fbc-ba60-ed46327e75c4 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:56.868830 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc8791-ea70-4fbc-ba60-ed46327e75c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:56.869249 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc8791-ea70-4fbc-ba60-ed46327e75c4 admin admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:56.870194 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fc8791-ea70-4fbc-ba60-ed46327e75c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:50:56.872858 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fc8791-ea70-4fbc-ba60-ed46327e75c4 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:50:56.909101 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1147/2289] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:50:56 2022] POST /identity/v3/users => generated 290 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:50:56.911067 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3382a898-3efe-4533-afb4-874cde8d0609 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:50:56.922691 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3382a898-3efe-4533-afb4-874cde8d0609 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:50:56.928672 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726a6717-2ae5-43f6-86cf-6d9e4df59db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:50:56.929783 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726a6717-2ae5-43f6-86cf-6d9e4df59db5 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:56.930063 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726a6717-2ae5-43f6-86cf-6d9e4df59db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:56.930333 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726a6717-2ae5-43f6-86cf-6d9e4df59db5 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:56.930977 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726a6717-2ae5-43f6-86cf-6d9e4df59db5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:50:56.932675 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726a6717-2ae5-43f6-86cf-6d9e4df59db5 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:50:56.948281 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1143/2290] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:50:56 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:50:56.949928 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fc8791-ea70-4fbc-ba60-ed46327e75c4 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:50:56.963785 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fc8791-ea70-4fbc-ba60-ed46327e75c4 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:50:56.970835 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc6bfc3-9387-4608-a572-28230cb8f9fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:50:56.971731 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc6bfc3-9387-4608-a572-28230cb8f9fd admin admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:56.972326 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc6bfc3-9387-4608-a572-28230cb8f9fd admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:56.972605 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc6bfc3-9387-4608-a572-28230cb8f9fd admin admin] PATH_INFO: `/v3/projects/2e60624dc78244f0b5b88fdc4fddd23d/users/617dab37627c4f23bc91613aeab15431/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:56.992692 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc6bfc3-9387-4608-a572-28230cb8f9fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e60624dc78244f0b5b88fdc4fddd23d, user_id=617dab37627c4f23bc91613aeab15431, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:50:56.995927 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc6bfc3-9387-4608-a572-28230cb8f9fd admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:50:57.012919 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1148/2291] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:50:56 2022] PUT /identity/v3/projects/2e60624dc78244f0b5b88fdc4fddd23d/users/617dab37627c4f23bc91613aeab15431/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 10:50:57.017689 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726a6717-2ae5-43f6-86cf-6d9e4df59db5 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:50:57.028041 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726a6717-2ae5-43f6-86cf-6d9e4df59db5 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:50:57.093775 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ef397a-9c0d-4786-80c8-3d2292876143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:50:57.094571 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ef397a-9c0d-4786-80c8-3d2292876143 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:57.095170 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ef397a-9c0d-4786-80c8-3d2292876143 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:57.095657 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ef397a-9c0d-4786-80c8-3d2292876143 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:57.096873 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ef397a-9c0d-4786-80c8-3d2292876143 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:50:57.098721 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ef397a-9c0d-4786-80c8-3d2292876143 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:50:57.115255 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1144/2292] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:50:57 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 10:50:57.116526 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc6bfc3-9387-4608-a572-28230cb8f9fd admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:50:57.128228 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc6bfc3-9387-4608-a572-28230cb8f9fd admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:50:57.135045 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa6a592-dd9c-4200-a049-5e158b90db7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:50:57.135750 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa6a592-dd9c-4200-a049-5e158b90db7b admin admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:57.136029 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa6a592-dd9c-4200-a049-5e158b90db7b admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:57.136303 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa6a592-dd9c-4200-a049-5e158b90db7b admin admin] PATH_INFO: `/v3/projects/2e60624dc78244f0b5b88fdc4fddd23d/users/617dab37627c4f23bc91613aeab15431/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:57.138216 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa6a592-dd9c-4200-a049-5e158b90db7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e60624dc78244f0b5b88fdc4fddd23d, user_id=617dab37627c4f23bc91613aeab15431, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:50:57.141578 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa6a592-dd9c-4200-a049-5e158b90db7b admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:50:57.153472 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1149/2293] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:50:57 2022] PUT /identity/v3/projects/2e60624dc78244f0b5b88fdc4fddd23d/users/617dab37627c4f23bc91613aeab15431/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:50:57.165061 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5980b379-5267-445e-9c9b-783bca6db8ec None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:57.165641 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5980b379-5267-445e-9c9b-783bca6db8ec None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:57.165954 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5980b379-5267-445e-9c9b-783bca6db8ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:57.201301 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-5980b379-5267-445e-9c9b-783bca6db8ec None None] MFA Rules not processed for user `617dab37627c4f23bc91613aeab15431`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:50:57.203027 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-5980b379-5267-445e-9c9b-783bca6db8ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:50:57.294531 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1145/2294] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:50:57 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 10:50:57.318740 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa6a592-dd9c-4200-a049-5e158b90db7b admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:50:57.321614 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-aaa6a592-dd9c-4200-a049-5e158b90db7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:50:57.415203 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa6a592-dd9c-4200-a049-5e158b90db7b admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:50:57.425035 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32c244f-3238-46b9-b847-9b569d0260f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:50:57.425930 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c244f-3238-46b9-b847-9b569d0260f1 service cinder] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:57.426316 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c244f-3238-46b9-b847-9b569d0260f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:57.426814 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c244f-3238-46b9-b847-9b569d0260f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:57.438588 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32c244f-3238-46b9-b847-9b569d0260f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:50:57.440299 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32c244f-3238-46b9-b847-9b569d0260f1 service cinder] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:50:57.455290 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1150/2295] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:50:57 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:50:57.518351 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5980b379-5267-445e-9c9b-783bca6db8ec None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:50:57.520191 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-5980b379-5267-445e-9c9b-783bca6db8ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:50:57.536402 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5980b379-5267-445e-9c9b-783bca6db8ec None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:50:57.541196 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7ba32d-98cd-410e-b82c-9ad942f4fd2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:50:57.542157 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7ba32d-98cd-410e-b82c-9ad942f4fd2f service cinder] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:50:57.542680 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7ba32d-98cd-410e-b82c-9ad942f4fd2f service cinder] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:50:57.543087 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7ba32d-98cd-410e-b82c-9ad942f4fd2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:50:57.596201 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:50:57.629668 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7ba32d-98cd-410e-b82c-9ad942f4fd2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:50:57.631644 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7ba32d-98cd-410e-b82c-9ad942f4fd2f service cinder] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:50:57.659052 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1146/2296] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:50:57 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:51:01.545982 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32c244f-3238-46b9-b847-9b569d0260f1 service cinder] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:01.621484 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32c244f-3238-46b9-b847-9b569d0260f1 service cinder] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:01.626934 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6370693-3355-4096-989e-d97791f3dee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:01.627609 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6370693-3355-4096-989e-d97791f3dee1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:01.627887 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6370693-3355-4096-989e-d97791f3dee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:01.628179 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6370693-3355-4096-989e-d97791f3dee1 admin admin] PATH_INFO: `/v3/users/97f5e57b7b304f78a84ce2305645eb30` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:01.629419 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6370693-3355-4096-989e-d97791f3dee1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97f5e57b7b304f78a84ce2305645eb30)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:01.632047 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6370693-3355-4096-989e-d97791f3dee1 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:01.697404 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:51:01.719182 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-b6370693-3355-4096-989e-d97791f3dee1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97f5e57b7b304f78a84ce2305645eb30'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:01.746093 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-b6370693-3355-4096-989e-d97791f3dee1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97f5e57b7b304f78a84ce2305645eb30'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:01.746093 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-b6370693-3355-4096-989e-d97791f3dee1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97f5e57b7b304f78a84ce2305645eb30'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:01.746093 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-b6370693-3355-4096-989e-d97791f3dee1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97f5e57b7b304f78a84ce2305645eb30'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:01.761726 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-b6370693-3355-4096-989e-d97791f3dee1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97f5e57b7b304f78a84ce2305645eb30'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:01.776527 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1151/2297] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:51:01 2022] DELETE /identity/v3/users/97f5e57b7b304f78a84ce2305645eb30 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:51:01.792779 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a1a80e-dfce-446e-8662-585984f90cdc None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:01.792779 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a1a80e-dfce-446e-8662-585984f90cdc None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:01.798141 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a1a80e-dfce-446e-8662-585984f90cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:01.824468 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-49a1a80e-dfce-446e-8662-585984f90cdc None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:51:01.825822 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-49a1a80e-dfce-446e-8662-585984f90cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:01.910605 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1147/2298] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:51:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:51:01.921349 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6370693-3355-4096-989e-d97791f3dee1 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:01.922904 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-b6370693-3355-4096-989e-d97791f3dee1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:01.999300 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6370693-3355-4096-989e-d97791f3dee1 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:02.003105 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae1ff83-02db-4287-8545-ba1fd5a6c395 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:02.003762 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae1ff83-02db-4287-8545-ba1fd5a6c395 service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:02.004047 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae1ff83-02db-4287-8545-ba1fd5a6c395 service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:02.004345 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae1ff83-02db-4287-8545-ba1fd5a6c395 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:02.014841 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae1ff83-02db-4287-8545-ba1fd5a6c395 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:02.016382 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae1ff83-02db-4287-8545-ba1fd5a6c395 service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:02.027838 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1152/2299] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:51:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:51:02.380157 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a1a80e-dfce-446e-8662-585984f90cdc None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:02.381700 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-49a1a80e-dfce-446e-8662-585984f90cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:02.398082 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a1a80e-dfce-446e-8662-585984f90cdc None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:02.404398 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8ee1a0-602c-43d0-8f6c-30ed13d0e4f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:02.405134 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ee1a0-602c-43d0-8f6c-30ed13d0e4f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:02.405416 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ee1a0-602c-43d0-8f6c-30ed13d0e4f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:02.405689 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ee1a0-602c-43d0-8f6c-30ed13d0e4f8 admin admin] PATH_INFO: `/v3/projects/b29a24009b4d47829d7f9acb064230c1` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:02.407374 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8ee1a0-602c-43d0-8f6c-30ed13d0e4f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b29a24009b4d47829d7f9acb064230c1)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:02.409407 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8ee1a0-602c-43d0-8f6c-30ed13d0e4f8 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:02.521929 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-ca8ee1a0-602c-43d0-8f6c-30ed13d0e4f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b29a24009b4d47829d7f9acb064230c1', 'request_id': 'req-ca8ee1a0-602c-43d0-8f6c-30ed13d0e4f8'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:02.528943 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-ca8ee1a0-602c-43d0-8f6c-30ed13d0e4f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b29a24009b4d47829d7f9acb064230c1', 'request_id': 'req-ca8ee1a0-602c-43d0-8f6c-30ed13d0e4f8'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:02.537682 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-ca8ee1a0-602c-43d0-8f6c-30ed13d0e4f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b29a24009b4d47829d7f9acb064230c1', 'request_id': 'req-ca8ee1a0-602c-43d0-8f6c-30ed13d0e4f8'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:02.550104 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-ca8ee1a0-602c-43d0-8f6c-30ed13d0e4f8 admin admin] The token cache is being invalidate because project b29a24009b4d47829d7f9acb064230c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90653) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:51:02.550392 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-ca8ee1a0-602c-43d0-8f6c-30ed13d0e4f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:02.551046 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-ca8ee1a0-602c-43d0-8f6c-30ed13d0e4f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:02.553459 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1148/2300] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:51:02 2022] DELETE /identity/v3/projects/b29a24009b4d47829d7f9acb064230c1 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:51:02.557472 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae1ff83-02db-4287-8545-ba1fd5a6c395 service neutron] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:02.574695 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae1ff83-02db-4287-8545-ba1fd5a6c395 service neutron] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:02.639088 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c37c73-6591-4487-b558-8fdc1ea4a119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:02.639941 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c37c73-6591-4487-b558-8fdc1ea4a119 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:02.640243 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c37c73-6591-4487-b558-8fdc1ea4a119 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:02.640539 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c37c73-6591-4487-b558-8fdc1ea4a119 admin admin] PATH_INFO: `/v3/users/617dab37627c4f23bc91613aeab15431` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:02.641736 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c37c73-6591-4487-b558-8fdc1ea4a119 admin admin] RBAC: Authorizing `identity:delete_user(user_id=617dab37627c4f23bc91613aeab15431)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:02.644063 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c37c73-6591-4487-b558-8fdc1ea4a119 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:02.700476 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-19c37c73-6591-4487-b558-8fdc1ea4a119 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '617dab37627c4f23bc91613aeab15431'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:02.708704 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-19c37c73-6591-4487-b558-8fdc1ea4a119 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '617dab37627c4f23bc91613aeab15431'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:02.709512 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-19c37c73-6591-4487-b558-8fdc1ea4a119 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '617dab37627c4f23bc91613aeab15431'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:02.710130 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-19c37c73-6591-4487-b558-8fdc1ea4a119 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '617dab37627c4f23bc91613aeab15431'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:02.716152 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:51:02.727295 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-19c37c73-6591-4487-b558-8fdc1ea4a119 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '617dab37627c4f23bc91613aeab15431'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:02.741914 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1153/2301] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:51:02 2022] DELETE /identity/v3/users/617dab37627c4f23bc91613aeab15431 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:51:03.087299 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8ee1a0-602c-43d0-8f6c-30ed13d0e4f8 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:03.088966 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-ca8ee1a0-602c-43d0-8f6c-30ed13d0e4f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:03.114901 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8ee1a0-602c-43d0-8f6c-30ed13d0e4f8 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:03.201889 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:51:03.211696 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e64d1d-f114-40a5-988b-1f3b61ffcfd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:03.212407 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e64d1d-f114-40a5-988b-1f3b61ffcfd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:03.212973 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e64d1d-f114-40a5-988b-1f3b61ffcfd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:03.213831 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e64d1d-f114-40a5-988b-1f3b61ffcfd2 admin admin] PATH_INFO: `/v3/projects/2e60624dc78244f0b5b88fdc4fddd23d` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:03.215208 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e64d1d-f114-40a5-988b-1f3b61ffcfd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e60624dc78244f0b5b88fdc4fddd23d)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:03.218043 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e64d1d-f114-40a5-988b-1f3b61ffcfd2 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:03.277766 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-15e64d1d-f114-40a5-988b-1f3b61ffcfd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e60624dc78244f0b5b88fdc4fddd23d', 'request_id': 'req-15e64d1d-f114-40a5-988b-1f3b61ffcfd2'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:03.297215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-15e64d1d-f114-40a5-988b-1f3b61ffcfd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e60624dc78244f0b5b88fdc4fddd23d', 'request_id': 'req-15e64d1d-f114-40a5-988b-1f3b61ffcfd2'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:03.316807 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-15e64d1d-f114-40a5-988b-1f3b61ffcfd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e60624dc78244f0b5b88fdc4fddd23d', 'request_id': 'req-15e64d1d-f114-40a5-988b-1f3b61ffcfd2'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:03.331150 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-15e64d1d-f114-40a5-988b-1f3b61ffcfd2 admin admin] The token cache is being invalidate because project 2e60624dc78244f0b5b88fdc4fddd23d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90653) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:51:03.331585 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-15e64d1d-f114-40a5-988b-1f3b61ffcfd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:03.332257 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-15e64d1d-f114-40a5-988b-1f3b61ffcfd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:03.337822 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1149/2302] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:51:03 2022] DELETE /identity/v3/projects/2e60624dc78244f0b5b88fdc4fddd23d => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:51:03.340284 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ceb57-55fb-422e-820c-7334fc92a1ce None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:03.340703 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ceb57-55fb-422e-820c-7334fc92a1ce None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:03.341062 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ceb57-55fb-422e-820c-7334fc92a1ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:03.364677 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-c45ceb57-55fb-422e-820c-7334fc92a1ce None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:51:03.366018 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-c45ceb57-55fb-422e-820c-7334fc92a1ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:03.435194 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1154/2303] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:51:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:51:03.443878 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75121742-5d5c-4097-b8d1-5fde47145f6b None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:03.444163 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75121742-5d5c-4097-b8d1-5fde47145f6b None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:03.444437 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75121742-5d5c-4097-b8d1-5fde47145f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:03.467543 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-75121742-5d5c-4097-b8d1-5fde47145f6b None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:51:03.468746 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-75121742-5d5c-4097-b8d1-5fde47145f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:03.477576 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1150/2304] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:51:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:51:03.481790 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45ceb57-55fb-422e-820c-7334fc92a1ce None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:03.492699 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45ceb57-55fb-422e-820c-7334fc92a1ce None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:03.500360 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266af1db-ec2a-4ddf-93b6-303c3798cfd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:03.501083 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266af1db-ec2a-4ddf-93b6-303c3798cfd3 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:03.501464 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266af1db-ec2a-4ddf-93b6-303c3798cfd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:03.501836 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266af1db-ec2a-4ddf-93b6-303c3798cfd3 admin admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:03.502718 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266af1db-ec2a-4ddf-93b6-303c3798cfd3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:51:03.503123 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266af1db-ec2a-4ddf-93b6-303c3798cfd3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:03.504867 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266af1db-ec2a-4ddf-93b6-303c3798cfd3 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:03.530843 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1155/2305] 10.222.0.15 () {58 vars in 1153 bytes} [Sun Sep 11 10:51:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:51:03.535149 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75121742-5d5c-4097-b8d1-5fde47145f6b None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:03.545793 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75121742-5d5c-4097-b8d1-5fde47145f6b None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:03.551629 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626b2b34-7185-4f82-af35-83d18a7d611d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:03.552374 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626b2b34-7185-4f82-af35-83d18a7d611d admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:03.552753 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626b2b34-7185-4f82-af35-83d18a7d611d admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:03.553149 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626b2b34-7185-4f82-af35-83d18a7d611d admin admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:03.553939 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626b2b34-7185-4f82-af35-83d18a7d611d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:03.555972 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626b2b34-7185-4f82-af35-83d18a7d611d admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:03.601718 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1151/2306] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:51:03 2022] POST /identity/v3/projects => generated 381 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 10:51:03.603273 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266af1db-ec2a-4ddf-93b6-303c3798cfd3 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:03.617104 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266af1db-ec2a-4ddf-93b6-303c3798cfd3 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:03.689203 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf31cdb-9839-4a83-87df-f1c9820e9fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:03.690193 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf31cdb-9839-4a83-87df-f1c9820e9fa1 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:03.690897 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf31cdb-9839-4a83-87df-f1c9820e9fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:03.691558 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf31cdb-9839-4a83-87df-f1c9820e9fa1 admin admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:03.692425 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf31cdb-9839-4a83-87df-f1c9820e9fa1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:03.694509 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf31cdb-9839-4a83-87df-f1c9820e9fa1 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:03.724300 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1156/2307] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:51:03 2022] POST /identity/v3/users => generated 291 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 10:51:03.741313 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626b2b34-7185-4f82-af35-83d18a7d611d admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:03.754461 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626b2b34-7185-4f82-af35-83d18a7d611d admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:03.760182 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72fca61-7869-4a01-b8da-52d94ea82163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:03.761111 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72fca61-7869-4a01-b8da-52d94ea82163 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:03.761741 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72fca61-7869-4a01-b8da-52d94ea82163 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:03.762237 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72fca61-7869-4a01-b8da-52d94ea82163 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:03.763222 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72fca61-7869-4a01-b8da-52d94ea82163 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:03.765955 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72fca61-7869-4a01-b8da-52d94ea82163 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:03.787049 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1152/2308] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:51:03 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:51:03.790797 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf31cdb-9839-4a83-87df-f1c9820e9fa1 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:03.803168 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf31cdb-9839-4a83-87df-f1c9820e9fa1 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:03.808475 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4ab24a-947c-4fab-88a0-be0491d93b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:03.809173 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4ab24a-947c-4fab-88a0-be0491d93b15 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:03.809529 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4ab24a-947c-4fab-88a0-be0491d93b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:03.809874 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4ab24a-947c-4fab-88a0-be0491d93b15 admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:03.811260 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4ab24a-947c-4fab-88a0-be0491d93b15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:03.816322 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4ab24a-947c-4fab-88a0-be0491d93b15 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:03.831455 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1157/2309] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:51:03 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:51:03.835928 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72fca61-7869-4a01-b8da-52d94ea82163 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:03.847470 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72fca61-7869-4a01-b8da-52d94ea82163 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:03.852750 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a0f96d-e339-433d-b215-34a53078b1b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:03.853603 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a0f96d-e339-433d-b215-34a53078b1b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:03.854060 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a0f96d-e339-433d-b215-34a53078b1b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:03.854511 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a0f96d-e339-433d-b215-34a53078b1b9 admin admin] PATH_INFO: `/v3/projects/8a8f9ffc68644eb889df3bc1d9978be0/users/0c1b071bbd1347329423d8570adcb381/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:03.872728 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a0f96d-e339-433d-b215-34a53078b1b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a8f9ffc68644eb889df3bc1d9978be0, user_id=0c1b071bbd1347329423d8570adcb381, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:03.874833 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a0f96d-e339-433d-b215-34a53078b1b9 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:03.892679 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1153/2310] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:51:03 2022] PUT /identity/v3/projects/8a8f9ffc68644eb889df3bc1d9978be0/users/0c1b071bbd1347329423d8570adcb381/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 10:51:03.905298 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab25cdb-7c55-48c5-8830-21e4a4992dea None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:03.905628 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab25cdb-7c55-48c5-8830-21e4a4992dea None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:03.905967 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab25cdb-7c55-48c5-8830-21e4a4992dea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:03.941540 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-0ab25cdb-7c55-48c5-8830-21e4a4992dea None None] MFA Rules not processed for user `0c1b071bbd1347329423d8570adcb381`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:51:03.943044 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-0ab25cdb-7c55-48c5-8830-21e4a4992dea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:03.987982 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:51:04.026202 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1158/2311] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:51:03 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:51:04.031811 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a0f96d-e339-433d-b215-34a53078b1b9 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:04.100986 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a0f96d-e339-433d-b215-34a53078b1b9 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:04.106334 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69af09cc-5f7d-406a-9dce-6b84840d99ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:04.106885 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af09cc-5f7d-406a-9dce-6b84840d99ca admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:04.107167 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af09cc-5f7d-406a-9dce-6b84840d99ca admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:04.108051 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af09cc-5f7d-406a-9dce-6b84840d99ca admin admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:04.108686 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69af09cc-5f7d-406a-9dce-6b84840d99ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:04.110643 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69af09cc-5f7d-406a-9dce-6b84840d99ca admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:04.145498 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1154/2312] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:51:04 2022] POST /identity/v3/projects => generated 381 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:51:04.150170 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab25cdb-7c55-48c5-8830-21e4a4992dea None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:04.162984 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab25cdb-7c55-48c5-8830-21e4a4992dea None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:04.267623 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575d2c91-2c7d-46d3-8d88-831c61e0d69f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:04.268551 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575d2c91-2c7d-46d3-8d88-831c61e0d69f admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:04.268845 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575d2c91-2c7d-46d3-8d88-831c61e0d69f admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:04.269126 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575d2c91-2c7d-46d3-8d88-831c61e0d69f admin admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:04.269887 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575d2c91-2c7d-46d3-8d88-831c61e0d69f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:04.272149 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575d2c91-2c7d-46d3-8d88-831c61e0d69f admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:04.299465 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1159/2313] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:51:04 2022] POST /identity/v3/users => generated 297 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:51:04.302063 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69af09cc-5f7d-406a-9dce-6b84840d99ca admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:04.314641 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69af09cc-5f7d-406a-9dce-6b84840d99ca admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:04.319762 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5ecdba-22ac-46ae-996c-bf4cb58a4328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:04.320458 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5ecdba-22ac-46ae-996c-bf4cb58a4328 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:04.320837 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5ecdba-22ac-46ae-996c-bf4cb58a4328 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:04.321213 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5ecdba-22ac-46ae-996c-bf4cb58a4328 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:04.321931 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5ecdba-22ac-46ae-996c-bf4cb58a4328 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:04.325877 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5ecdba-22ac-46ae-996c-bf4cb58a4328 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:04.328975 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:51:04.340922 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1155/2314] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:51:04 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:51:04.345549 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575d2c91-2c7d-46d3-8d88-831c61e0d69f admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:04.377666 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575d2c91-2c7d-46d3-8d88-831c61e0d69f admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:04.385883 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2057abd4-8131-4d16-99b1-60e1d0c19afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:04.387121 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2057abd4-8131-4d16-99b1-60e1d0c19afe admin admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:04.387795 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2057abd4-8131-4d16-99b1-60e1d0c19afe admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:04.388420 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2057abd4-8131-4d16-99b1-60e1d0c19afe admin admin] PATH_INFO: `/v3/projects/32c71bd7669a4c7ea2c722c26174f66f/users/6cbe36c539494bb19014261de8b2a0c3/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:04.412727 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2057abd4-8131-4d16-99b1-60e1d0c19afe admin admin] RBAC: Authorizing `identity:create_grant(project_id=32c71bd7669a4c7ea2c722c26174f66f, user_id=6cbe36c539494bb19014261de8b2a0c3, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:04.416334 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2057abd4-8131-4d16-99b1-60e1d0c19afe admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:04.434839 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1160/2315] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:51:04 2022] PUT /identity/v3/projects/32c71bd7669a4c7ea2c722c26174f66f/users/6cbe36c539494bb19014261de8b2a0c3/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:51:04.440233 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5ecdba-22ac-46ae-996c-bf4cb58a4328 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:04.454787 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5ecdba-22ac-46ae-996c-bf4cb58a4328 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:04.525507 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f866743-17f7-4f0f-813c-b2645fd51861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:04.525787 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f866743-17f7-4f0f-813c-b2645fd51861 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:04.526164 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f866743-17f7-4f0f-813c-b2645fd51861 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:04.526537 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f866743-17f7-4f0f-813c-b2645fd51861 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:04.527481 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f866743-17f7-4f0f-813c-b2645fd51861 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:04.529308 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f866743-17f7-4f0f-813c-b2645fd51861 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:04.541846 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1156/2316] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:51:04 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 10:51:04.547751 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2057abd4-8131-4d16-99b1-60e1d0c19afe admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:04.562371 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2057abd4-8131-4d16-99b1-60e1d0c19afe admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:04.568089 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113c641d-5d0b-4679-916b-b09023723163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:04.569150 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113c641d-5d0b-4679-916b-b09023723163 admin admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:04.569432 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113c641d-5d0b-4679-916b-b09023723163 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:04.569784 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113c641d-5d0b-4679-916b-b09023723163 admin admin] PATH_INFO: `/v3/projects/32c71bd7669a4c7ea2c722c26174f66f/users/6cbe36c539494bb19014261de8b2a0c3/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:04.572663 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113c641d-5d0b-4679-916b-b09023723163 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32c71bd7669a4c7ea2c722c26174f66f, user_id=6cbe36c539494bb19014261de8b2a0c3, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:04.576646 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113c641d-5d0b-4679-916b-b09023723163 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:04.590202 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1161/2317] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:51:04 2022] PUT /identity/v3/projects/32c71bd7669a4c7ea2c722c26174f66f/users/6cbe36c539494bb19014261de8b2a0c3/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:51:04.601337 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f234925-35b1-47eb-88d9-c44c2fb3a135 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:04.601747 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f234925-35b1-47eb-88d9-c44c2fb3a135 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:04.602159 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f234925-35b1-47eb-88d9-c44c2fb3a135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:04.636313 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-0f234925-35b1-47eb-88d9-c44c2fb3a135 None None] MFA Rules not processed for user `6cbe36c539494bb19014261de8b2a0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:51:04.638169 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-0f234925-35b1-47eb-88d9-c44c2fb3a135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:04.734396 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1157/2318] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:51:04 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:51:04.762554 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113c641d-5d0b-4679-916b-b09023723163 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:04.765184 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-113c641d-5d0b-4679-916b-b09023723163 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:04.929360 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113c641d-5d0b-4679-916b-b09023723163 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:04.933999 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18ecb0c-c738-47fe-a4ec-1e42619398d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:04.934648 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ecb0c-c738-47fe-a4ec-1e42619398d7 service cinder] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:04.934979 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ecb0c-c738-47fe-a4ec-1e42619398d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:04.935227 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ecb0c-c738-47fe-a4ec-1e42619398d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:04.950397 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18ecb0c-c738-47fe-a4ec-1e42619398d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:04.952644 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18ecb0c-c738-47fe-a4ec-1e42619398d7 service cinder] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:04.965018 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1162/2319] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:51:04 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:51:05.254423 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f234925-35b1-47eb-88d9-c44c2fb3a135 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:05.256197 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-0f234925-35b1-47eb-88d9-c44c2fb3a135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:05.381927 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f234925-35b1-47eb-88d9-c44c2fb3a135 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:05.387046 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6034e9-4745-4661-a5c5-42925454221e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:05.387826 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6034e9-4745-4661-a5c5-42925454221e service neutron] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:05.388221 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6034e9-4745-4661-a5c5-42925454221e service neutron] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:05.389220 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6034e9-4745-4661-a5c5-42925454221e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:05.391874 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-8b6034e9-4745-4661-a5c5-42925454221e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:05.412700 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18ecb0c-c738-47fe-a4ec-1e42619398d7 service cinder] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:05.426277 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18ecb0c-c738-47fe-a4ec-1e42619398d7 service cinder] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:05.431247 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a1fc12-2ce1-43ac-ad3d-6502a294f126 tempest-VolumeQuotaClassesTest-1358458933 tempest-VolumeQuotaClassesTest-1358458933-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6cbe36c539494bb19014261de8b2a0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32c71bd7669a4c7ea2c722c26174f66f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:05.431907 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a1fc12-2ce1-43ac-ad3d-6502a294f126 tempest-VolumeQuotaClassesTest-1358458933 tempest-VolumeQuotaClassesTest-1358458933-project-admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:05.432196 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a1fc12-2ce1-43ac-ad3d-6502a294f126 tempest-VolumeQuotaClassesTest-1358458933 tempest-VolumeQuotaClassesTest-1358458933-project-admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:05.432695 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a1fc12-2ce1-43ac-ad3d-6502a294f126 tempest-VolumeQuotaClassesTest-1358458933 tempest-VolumeQuotaClassesTest-1358458933-project-admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:05.433349 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a1fc12-2ce1-43ac-ad3d-6502a294f126 tempest-VolumeQuotaClassesTest-1358458933 tempest-VolumeQuotaClassesTest-1358458933-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:51:05.433651 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a1fc12-2ce1-43ac-ad3d-6502a294f126 tempest-VolumeQuotaClassesTest-1358458933 tempest-VolumeQuotaClassesTest-1358458933-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:05.435271 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a1fc12-2ce1-43ac-ad3d-6502a294f126 tempest-VolumeQuotaClassesTest-1358458933 tempest-VolumeQuotaClassesTest-1358458933-project-admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:05.444337 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:51:05.449985 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1163/2320] 10.222.0.15 () {58 vars in 1153 bytes} [Sun Sep 11 10:51:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:51:05.455220 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a1fc12-2ce1-43ac-ad3d-6502a294f126 tempest-VolumeQuotaClassesTest-1358458933 tempest-VolumeQuotaClassesTest-1358458933-project-admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:05.466970 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a1fc12-2ce1-43ac-ad3d-6502a294f126 tempest-VolumeQuotaClassesTest-1358458933 tempest-VolumeQuotaClassesTest-1358458933-project-admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:05.473314 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb78d66-0b38-4260-bcd7-387c3c7fff5b tempest-VolumeQuotaClassesTest-1358458933 tempest-VolumeQuotaClassesTest-1358458933-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6cbe36c539494bb19014261de8b2a0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32c71bd7669a4c7ea2c722c26174f66f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:05.473971 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb78d66-0b38-4260-bcd7-387c3c7fff5b tempest-VolumeQuotaClassesTest-1358458933 tempest-VolumeQuotaClassesTest-1358458933-project-admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:05.474208 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb78d66-0b38-4260-bcd7-387c3c7fff5b tempest-VolumeQuotaClassesTest-1358458933 tempest-VolumeQuotaClassesTest-1358458933-project-admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:05.474562 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb78d66-0b38-4260-bcd7-387c3c7fff5b tempest-VolumeQuotaClassesTest-1358458933 tempest-VolumeQuotaClassesTest-1358458933-project-admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:05.475102 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb78d66-0b38-4260-bcd7-387c3c7fff5b tempest-VolumeQuotaClassesTest-1358458933 tempest-VolumeQuotaClassesTest-1358458933-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:05.477117 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb78d66-0b38-4260-bcd7-387c3c7fff5b tempest-VolumeQuotaClassesTest-1358458933 tempest-VolumeQuotaClassesTest-1358458933-project-admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:05.489791 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6034e9-4745-4661-a5c5-42925454221e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:05.491883 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6034e9-4745-4661-a5c5-42925454221e service neutron] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:05.494094 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:51:05.506195 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1158/2321] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:51:05 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:51:05.519649 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1164/2322] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:51:05 2022] POST /identity/v3/projects => generated 354 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:51:05.521043 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6034e9-4745-4661-a5c5-42925454221e service neutron] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:05.537479 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6034e9-4745-4661-a5c5-42925454221e service neutron] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:05.664981 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de912ce1-6373-4875-a33e-5c07fa2c0d2f tempest-VolumeQuotaClassesTest-1358458933 tempest-VolumeQuotaClassesTest-1358458933-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6cbe36c539494bb19014261de8b2a0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32c71bd7669a4c7ea2c722c26174f66f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:05.666232 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de912ce1-6373-4875-a33e-5c07fa2c0d2f tempest-VolumeQuotaClassesTest-1358458933 tempest-VolumeQuotaClassesTest-1358458933-project-admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:05.667062 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de912ce1-6373-4875-a33e-5c07fa2c0d2f tempest-VolumeQuotaClassesTest-1358458933 tempest-VolumeQuotaClassesTest-1358458933-project-admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:05.667677 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de912ce1-6373-4875-a33e-5c07fa2c0d2f tempest-VolumeQuotaClassesTest-1358458933 tempest-VolumeQuotaClassesTest-1358458933-project-admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:05.668535 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de912ce1-6373-4875-a33e-5c07fa2c0d2f tempest-VolumeQuotaClassesTest-1358458933 tempest-VolumeQuotaClassesTest-1358458933-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:51:05.669140 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de912ce1-6373-4875-a33e-5c07fa2c0d2f tempest-VolumeQuotaClassesTest-1358458933 tempest-VolumeQuotaClassesTest-1358458933-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:05.671995 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de912ce1-6373-4875-a33e-5c07fa2c0d2f tempest-VolumeQuotaClassesTest-1358458933 tempest-VolumeQuotaClassesTest-1358458933-project-admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:05.695683 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1159/2323] 10.222.0.15 () {58 vars in 1153 bytes} [Sun Sep 11 10:51:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:51:05.749254 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb78d66-0b38-4260-bcd7-387c3c7fff5b tempest-VolumeQuotaClassesTest-1358458933 tempest-VolumeQuotaClassesTest-1358458933-project-admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:05.762755 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb78d66-0b38-4260-bcd7-387c3c7fff5b tempest-VolumeQuotaClassesTest-1358458933 tempest-VolumeQuotaClassesTest-1358458933-project-admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:05.774036 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d96819-1b57-4056-873d-d6503dee7e07 tempest-VolumeQuotaClassesTest-1358458933 tempest-VolumeQuotaClassesTest-1358458933-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6cbe36c539494bb19014261de8b2a0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32c71bd7669a4c7ea2c722c26174f66f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:05.774853 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d96819-1b57-4056-873d-d6503dee7e07 tempest-VolumeQuotaClassesTest-1358458933 tempest-VolumeQuotaClassesTest-1358458933-project-admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:05.775206 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d96819-1b57-4056-873d-d6503dee7e07 tempest-VolumeQuotaClassesTest-1358458933 tempest-VolumeQuotaClassesTest-1358458933-project-admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:05.775581 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d96819-1b57-4056-873d-d6503dee7e07 tempest-VolumeQuotaClassesTest-1358458933 tempest-VolumeQuotaClassesTest-1358458933-project-admin] PATH_INFO: `/v3/projects/cb723eba2fa34fa88ec125c5de438dec` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:05.779968 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d96819-1b57-4056-873d-d6503dee7e07 tempest-VolumeQuotaClassesTest-1358458933 tempest-VolumeQuotaClassesTest-1358458933-project-admin] RBAC: Authorizing `identity:delete_project(project_id=cb723eba2fa34fa88ec125c5de438dec)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:05.783248 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d96819-1b57-4056-873d-d6503dee7e07 tempest-VolumeQuotaClassesTest-1358458933 tempest-VolumeQuotaClassesTest-1358458933-project-admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:05.851210 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-35d96819-1b57-4056-873d-d6503dee7e07 tempest-VolumeQuotaClassesTest-1358458933 tempest-VolumeQuotaClassesTest-1358458933-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb723eba2fa34fa88ec125c5de438dec', 'request_id': 'req-35d96819-1b57-4056-873d-d6503dee7e07'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:05.860793 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-35d96819-1b57-4056-873d-d6503dee7e07 tempest-VolumeQuotaClassesTest-1358458933 tempest-VolumeQuotaClassesTest-1358458933-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb723eba2fa34fa88ec125c5de438dec', 'request_id': 'req-35d96819-1b57-4056-873d-d6503dee7e07'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:05.868675 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-35d96819-1b57-4056-873d-d6503dee7e07 tempest-VolumeQuotaClassesTest-1358458933 tempest-VolumeQuotaClassesTest-1358458933-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb723eba2fa34fa88ec125c5de438dec', 'request_id': 'req-35d96819-1b57-4056-873d-d6503dee7e07'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:05.889224 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-35d96819-1b57-4056-873d-d6503dee7e07 tempest-VolumeQuotaClassesTest-1358458933 tempest-VolumeQuotaClassesTest-1358458933-project-admin] The token cache is being invalidate because project cb723eba2fa34fa88ec125c5de438dec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:51:05.889610 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-35d96819-1b57-4056-873d-d6503dee7e07 tempest-VolumeQuotaClassesTest-1358458933 tempest-VolumeQuotaClassesTest-1358458933-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:05.890662 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-35d96819-1b57-4056-873d-d6503dee7e07 tempest-VolumeQuotaClassesTest-1358458933 tempest-VolumeQuotaClassesTest-1358458933-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:05.893266 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1165/2324] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:51:05 2022] DELETE /identity/v3/projects/cb723eba2fa34fa88ec125c5de438dec => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:51:06.199709 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de912ce1-6373-4875-a33e-5c07fa2c0d2f tempest-VolumeQuotaClassesTest-1358458933 tempest-VolumeQuotaClassesTest-1358458933-project-admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:06.201546 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-de912ce1-6373-4875-a33e-5c07fa2c0d2f tempest-VolumeQuotaClassesTest-1358458933 tempest-VolumeQuotaClassesTest-1358458933-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:06.293852 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de912ce1-6373-4875-a33e-5c07fa2c0d2f tempest-VolumeQuotaClassesTest-1358458933 tempest-VolumeQuotaClassesTest-1358458933-project-admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:06.380144 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e89f67a-ffa9-4d7d-9a6e-ff78c2264586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:06.380802 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e89f67a-ffa9-4d7d-9a6e-ff78c2264586 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:06.381083 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e89f67a-ffa9-4d7d-9a6e-ff78c2264586 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:06.381357 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e89f67a-ffa9-4d7d-9a6e-ff78c2264586 admin admin] PATH_INFO: `/v3/users/0c1b071bbd1347329423d8570adcb381` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:06.383531 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e89f67a-ffa9-4d7d-9a6e-ff78c2264586 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c1b071bbd1347329423d8570adcb381)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:06.392847 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e89f67a-ffa9-4d7d-9a6e-ff78c2264586 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:06.421057 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:51:06.466876 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-4e89f67a-ffa9-4d7d-9a6e-ff78c2264586 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c1b071bbd1347329423d8570adcb381'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:06.477181 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-4e89f67a-ffa9-4d7d-9a6e-ff78c2264586 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c1b071bbd1347329423d8570adcb381'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:06.478087 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-4e89f67a-ffa9-4d7d-9a6e-ff78c2264586 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c1b071bbd1347329423d8570adcb381'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:06.479031 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-4e89f67a-ffa9-4d7d-9a6e-ff78c2264586 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c1b071bbd1347329423d8570adcb381'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:06.500728 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-4e89f67a-ffa9-4d7d-9a6e-ff78c2264586 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c1b071bbd1347329423d8570adcb381'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:06.533002 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1160/2325] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:51:06 2022] DELETE /identity/v3/users/0c1b071bbd1347329423d8570adcb381 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:51:06.534094 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2131340-a056-4671-baf9-bafae8ee7ada None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:06.534391 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2131340-a056-4671-baf9-bafae8ee7ada None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:06.534671 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2131340-a056-4671-baf9-bafae8ee7ada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:06.564985 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-a2131340-a056-4671-baf9-bafae8ee7ada None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:51:06.567712 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-a2131340-a056-4671-baf9-bafae8ee7ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:06.675467 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1166/2326] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:51:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 10:51:06.688605 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e89f67a-ffa9-4d7d-9a6e-ff78c2264586 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:06.691532 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-4e89f67a-ffa9-4d7d-9a6e-ff78c2264586 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:06.794553 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e89f67a-ffa9-4d7d-9a6e-ff78c2264586 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:06.799138 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19254f9-873b-4726-bde8-63563a4e1591 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:06.799788 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19254f9-873b-4726-bde8-63563a4e1591 service neutron] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:06.800090 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19254f9-873b-4726-bde8-63563a4e1591 service neutron] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:06.800351 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19254f9-873b-4726-bde8-63563a4e1591 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:06.811516 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19254f9-873b-4726-bde8-63563a4e1591 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:06.812961 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19254f9-873b-4726-bde8-63563a4e1591 service neutron] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:06.823609 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1161/2327] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:51:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 10:51:07.256178 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2131340-a056-4671-baf9-bafae8ee7ada None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:07.257681 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-a2131340-a056-4671-baf9-bafae8ee7ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:07.272590 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2131340-a056-4671-baf9-bafae8ee7ada None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:07.277784 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e93efb7-f6bf-47a9-a517-ac77d29deb3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:07.283995 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e93efb7-f6bf-47a9-a517-ac77d29deb3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:07.284375 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e93efb7-f6bf-47a9-a517-ac77d29deb3a admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:07.284597 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e93efb7-f6bf-47a9-a517-ac77d29deb3a admin admin] PATH_INFO: `/v3/projects/8a8f9ffc68644eb889df3bc1d9978be0` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:07.285859 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e93efb7-f6bf-47a9-a517-ac77d29deb3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a8f9ffc68644eb889df3bc1d9978be0)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:07.287838 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e93efb7-f6bf-47a9-a517-ac77d29deb3a admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:07.367027 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-8e93efb7-f6bf-47a9-a517-ac77d29deb3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a8f9ffc68644eb889df3bc1d9978be0', 'request_id': 'req-8e93efb7-f6bf-47a9-a517-ac77d29deb3a'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:07.376831 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-8e93efb7-f6bf-47a9-a517-ac77d29deb3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a8f9ffc68644eb889df3bc1d9978be0', 'request_id': 'req-8e93efb7-f6bf-47a9-a517-ac77d29deb3a'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:07.384628 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-8e93efb7-f6bf-47a9-a517-ac77d29deb3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a8f9ffc68644eb889df3bc1d9978be0', 'request_id': 'req-8e93efb7-f6bf-47a9-a517-ac77d29deb3a'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:07.401005 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:INSERT=3,keystone:SELECT=83,keystone:DELETE=14 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:51:07.409449 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-8e93efb7-f6bf-47a9-a517-ac77d29deb3a admin admin] The token cache is being invalidate because project 8a8f9ffc68644eb889df3bc1d9978be0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:51:07.409720 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-8e93efb7-f6bf-47a9-a517-ac77d29deb3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:07.429726 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-8e93efb7-f6bf-47a9-a517-ac77d29deb3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:07.440092 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1167/2328] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:51:07 2022] DELETE /identity/v3/projects/8a8f9ffc68644eb889df3bc1d9978be0 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 10:51:07.441142 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19254f9-873b-4726-bde8-63563a4e1591 service neutron] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:07.457861 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19254f9-873b-4726-bde8-63563a4e1591 service neutron] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:07.561108 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de01e4-6e41-4ecf-a56e-18c9f46d9b24 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:07.561747 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de01e4-6e41-4ecf-a56e-18c9f46d9b24 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:07.562441 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de01e4-6e41-4ecf-a56e-18c9f46d9b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:07.569313 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49fb173-f75f-41ef-9bd5-bb7408e81802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:07.570113 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49fb173-f75f-41ef-9bd5-bb7408e81802 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:07.570619 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49fb173-f75f-41ef-9bd5-bb7408e81802 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:07.571705 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49fb173-f75f-41ef-9bd5-bb7408e81802 admin admin] PATH_INFO: `/v3/users/6cbe36c539494bb19014261de8b2a0c3` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:07.573664 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49fb173-f75f-41ef-9bd5-bb7408e81802 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6cbe36c539494bb19014261de8b2a0c3)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:07.577969 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49fb173-f75f-41ef-9bd5-bb7408e81802 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:07.595986 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-30de01e4-6e41-4ecf-a56e-18c9f46d9b24 None None] MFA Rules not processed for user `e1b0815853b947bc8f50d91c1ce0ab5a`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:51:07.597513 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-30de01e4-6e41-4ecf-a56e-18c9f46d9b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:07.645371 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:51:07.661863 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-f49fb173-f75f-41ef-9bd5-bb7408e81802 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6cbe36c539494bb19014261de8b2a0c3'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:07.688916 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-f49fb173-f75f-41ef-9bd5-bb7408e81802 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6cbe36c539494bb19014261de8b2a0c3'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:07.689648 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-f49fb173-f75f-41ef-9bd5-bb7408e81802 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6cbe36c539494bb19014261de8b2a0c3'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:07.694892 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-f49fb173-f75f-41ef-9bd5-bb7408e81802 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6cbe36c539494bb19014261de8b2a0c3'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:07.711320 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-f49fb173-f75f-41ef-9bd5-bb7408e81802 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6cbe36c539494bb19014261de8b2a0c3'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:07.721887 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1168/2329] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:51:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 10:51:07.747311 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1162/2330] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:51:07 2022] DELETE /identity/v3/users/6cbe36c539494bb19014261de8b2a0c3 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:51:08.178502 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30de01e4-6e41-4ecf-a56e-18c9f46d9b24 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:08.180534 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-30de01e4-6e41-4ecf-a56e-18c9f46d9b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:08.223636 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30de01e4-6e41-4ecf-a56e-18c9f46d9b24 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:08.340441 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c9a6b6-1501-4c30-9c98-9a8d44e82917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:08.341167 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c9a6b6-1501-4c30-9c98-9a8d44e82917 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:08.341429 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c9a6b6-1501-4c30-9c98-9a8d44e82917 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:08.341932 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c9a6b6-1501-4c30-9c98-9a8d44e82917 admin admin] PATH_INFO: `/v3/projects/32c71bd7669a4c7ea2c722c26174f66f` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:08.343197 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c9a6b6-1501-4c30-9c98-9a8d44e82917 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32c71bd7669a4c7ea2c722c26174f66f)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:08.346062 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c9a6b6-1501-4c30-9c98-9a8d44e82917 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:08.431335 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-68c9a6b6-1501-4c30-9c98-9a8d44e82917 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32c71bd7669a4c7ea2c722c26174f66f', 'request_id': 'req-68c9a6b6-1501-4c30-9c98-9a8d44e82917'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:08.439512 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-68c9a6b6-1501-4c30-9c98-9a8d44e82917 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32c71bd7669a4c7ea2c722c26174f66f', 'request_id': 'req-68c9a6b6-1501-4c30-9c98-9a8d44e82917'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:08.446998 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-68c9a6b6-1501-4c30-9c98-9a8d44e82917 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32c71bd7669a4c7ea2c722c26174f66f', 'request_id': 'req-68c9a6b6-1501-4c30-9c98-9a8d44e82917'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:08.470560 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-68c9a6b6-1501-4c30-9c98-9a8d44e82917 admin admin] The token cache is being invalidate because project 32c71bd7669a4c7ea2c722c26174f66f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:51:08.471549 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-68c9a6b6-1501-4c30-9c98-9a8d44e82917 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:08.474417 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-68c9a6b6-1501-4c30-9c98-9a8d44e82917 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:08.476253 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1169/2331] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:51:08 2022] DELETE /identity/v3/projects/32c71bd7669a4c7ea2c722c26174f66f => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:51:08.484549 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710ddeab-d6d9-481b-a62a-afd401c75ba8 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:08.485074 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710ddeab-d6d9-481b-a62a-afd401c75ba8 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:08.485170 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710ddeab-d6d9-481b-a62a-afd401c75ba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:08.524261 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-710ddeab-d6d9-481b-a62a-afd401c75ba8 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:51:08.526289 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-710ddeab-d6d9-481b-a62a-afd401c75ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:08.642127 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1163/2332] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:51:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:51:08.651805 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accafb50-06a3-40e1-bb7e-9402fb4ce59b None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:08.652222 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accafb50-06a3-40e1-bb7e-9402fb4ce59b None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:08.652719 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accafb50-06a3-40e1-bb7e-9402fb4ce59b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:08.689708 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-accafb50-06a3-40e1-bb7e-9402fb4ce59b None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:51:08.690884 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-accafb50-06a3-40e1-bb7e-9402fb4ce59b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:08.705489 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710ddeab-d6d9-481b-a62a-afd401c75ba8 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:08.712864 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1170/2333] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:51:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:51:08.717234 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710ddeab-d6d9-481b-a62a-afd401c75ba8 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:08.723146 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fc5ad2-3bb8-4e7b-ba31-c45f237b0f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:08.724066 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc5ad2-3bb8-4e7b-ba31-c45f237b0f1a admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:08.724366 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc5ad2-3bb8-4e7b-ba31-c45f237b0f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:08.724970 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc5ad2-3bb8-4e7b-ba31-c45f237b0f1a admin admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:08.725645 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fc5ad2-3bb8-4e7b-ba31-c45f237b0f1a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:51:08.725968 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fc5ad2-3bb8-4e7b-ba31-c45f237b0f1a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:08.727623 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fc5ad2-3bb8-4e7b-ba31-c45f237b0f1a admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:08.749353 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accafb50-06a3-40e1-bb7e-9402fb4ce59b None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:08.759758 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accafb50-06a3-40e1-bb7e-9402fb4ce59b None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:08.760533 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1164/2334] 10.222.0.15 () {58 vars in 1153 bytes} [Sun Sep 11 10:51:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:51:08.768052 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf7c9e8-d2e9-45c0-a1f9-1db6e2e4b962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:08.771581 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf7c9e8-d2e9-45c0-a1f9-1db6e2e4b962 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:08.771893 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf7c9e8-d2e9-45c0-a1f9-1db6e2e4b962 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:08.772224 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf7c9e8-d2e9-45c0-a1f9-1db6e2e4b962 admin admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:08.772881 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf7c9e8-d2e9-45c0-a1f9-1db6e2e4b962 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:08.780172 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf7c9e8-d2e9-45c0-a1f9-1db6e2e4b962 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:08.840340 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1171/2335] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:51:08 2022] POST /identity/v3/projects => generated 385 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:51:08.844889 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fc5ad2-3bb8-4e7b-ba31-c45f237b0f1a admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:08.863060 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fc5ad2-3bb8-4e7b-ba31-c45f237b0f1a admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:09.002398 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba68fed-1573-4800-8e21-3541011fa653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:09.004880 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba68fed-1573-4800-8e21-3541011fa653 admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:09.005358 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba68fed-1573-4800-8e21-3541011fa653 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:09.005643 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba68fed-1573-4800-8e21-3541011fa653 admin admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:09.008575 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba68fed-1573-4800-8e21-3541011fa653 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:09.014463 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba68fed-1573-4800-8e21-3541011fa653 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:09.069197 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1165/2336] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:51:08 2022] POST /identity/v3/users => generated 293 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:51:09.070245 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf7c9e8-d2e9-45c0-a1f9-1db6e2e4b962 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:09.084847 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf7c9e8-d2e9-45c0-a1f9-1db6e2e4b962 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:09.101136 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb95eedb-b002-4e00-9c85-64b5fc3f385c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:09.104482 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb95eedb-b002-4e00-9c85-64b5fc3f385c admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:09.105147 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb95eedb-b002-4e00-9c85-64b5fc3f385c admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:09.105439 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb95eedb-b002-4e00-9c85-64b5fc3f385c admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:09.106184 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb95eedb-b002-4e00-9c85-64b5fc3f385c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:09.108658 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb95eedb-b002-4e00-9c85-64b5fc3f385c admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:09.119449 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:51:09.156839 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1172/2337] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:51:09.160617 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba68fed-1573-4800-8e21-3541011fa653 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:09.180482 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba68fed-1573-4800-8e21-3541011fa653 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:09.193036 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdd45c4-e68f-4cbc-8de8-d9a3c567b71d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:09.193612 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdd45c4-e68f-4cbc-8de8-d9a3c567b71d admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:09.194045 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdd45c4-e68f-4cbc-8de8-d9a3c567b71d admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:09.194383 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdd45c4-e68f-4cbc-8de8-d9a3c567b71d admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:09.195328 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdd45c4-e68f-4cbc-8de8-d9a3c567b71d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:09.202579 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdd45c4-e68f-4cbc-8de8-d9a3c567b71d admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:09.227385 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1166/2338] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:51:09.231550 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb95eedb-b002-4e00-9c85-64b5fc3f385c admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:09.254492 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb95eedb-b002-4e00-9c85-64b5fc3f385c admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:09.270805 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bba8bd-a288-44e2-92c8-6d02358b6426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:09.272760 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bba8bd-a288-44e2-92c8-6d02358b6426 admin admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:09.273227 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bba8bd-a288-44e2-92c8-6d02358b6426 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:09.273701 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bba8bd-a288-44e2-92c8-6d02358b6426 admin admin] PATH_INFO: `/v3/projects/155e70fb43d34b6d8cdfd16b7a318057/users/2d965d07755d4590accea12dbf84c46b/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:09.313604 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bba8bd-a288-44e2-92c8-6d02358b6426 admin admin] RBAC: Authorizing `identity:create_grant(project_id=155e70fb43d34b6d8cdfd16b7a318057, user_id=2d965d07755d4590accea12dbf84c46b, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:09.317742 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bba8bd-a288-44e2-92c8-6d02358b6426 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:09.384605 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1173/2339] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:51:09 2022] PUT /identity/v3/projects/155e70fb43d34b6d8cdfd16b7a318057/users/2d965d07755d4590accea12dbf84c46b/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:51:09.435824 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d2b3c9-1b7e-42ae-80f8-eedd2b63a83e None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:09.435824 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d2b3c9-1b7e-42ae-80f8-eedd2b63a83e None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:09.435824 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d2b3c9-1b7e-42ae-80f8-eedd2b63a83e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:09.485972 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-f1d2b3c9-1b7e-42ae-80f8-eedd2b63a83e None None] MFA Rules not processed for user `2d965d07755d4590accea12dbf84c46b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:51:09.487744 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-f1d2b3c9-1b7e-42ae-80f8-eedd2b63a83e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:09.507856 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:51:09.600310 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1167/2340] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:51:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:51:09.605526 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bba8bd-a288-44e2-92c8-6d02358b6426 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:09.714575 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bba8bd-a288-44e2-92c8-6d02358b6426 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:09.722543 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4f66b5-3c94-44ad-9c06-4665f1f83320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:09.725840 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4f66b5-3c94-44ad-9c06-4665f1f83320 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:09.727946 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4f66b5-3c94-44ad-9c06-4665f1f83320 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:09.728148 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4f66b5-3c94-44ad-9c06-4665f1f83320 admin admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:09.728964 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4f66b5-3c94-44ad-9c06-4665f1f83320 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:09.733808 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4f66b5-3c94-44ad-9c06-4665f1f83320 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:09.790417 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1174/2341] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:51:09 2022] POST /identity/v3/projects => generated 385 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:51:09.795293 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d2b3c9-1b7e-42ae-80f8-eedd2b63a83e None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:09.816052 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d2b3c9-1b7e-42ae-80f8-eedd2b63a83e None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:09.953342 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac601a2a-1c20-4964-9238-2782690cdf8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:09.954005 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac601a2a-1c20-4964-9238-2782690cdf8b admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:09.954822 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac601a2a-1c20-4964-9238-2782690cdf8b admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:09.955098 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac601a2a-1c20-4964-9238-2782690cdf8b admin admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:09.955761 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac601a2a-1c20-4964-9238-2782690cdf8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:09.957911 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac601a2a-1c20-4964-9238-2782690cdf8b admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:09.987580 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1168/2342] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:51:09 2022] POST /identity/v3/users => generated 293 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:51:09.991748 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4f66b5-3c94-44ad-9c06-4665f1f83320 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:10.005709 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4f66b5-3c94-44ad-9c06-4665f1f83320 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:10.012177 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04981e62-6b2a-4799-89be-3fd1de3dc84c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:10.013114 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04981e62-6b2a-4799-89be-3fd1de3dc84c admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:10.013708 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04981e62-6b2a-4799-89be-3fd1de3dc84c admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:10.014641 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04981e62-6b2a-4799-89be-3fd1de3dc84c admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:10.025598 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04981e62-6b2a-4799-89be-3fd1de3dc84c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:10.031548 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04981e62-6b2a-4799-89be-3fd1de3dc84c admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:10.055307 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1175/2343] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:51:10.060902 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac601a2a-1c20-4964-9238-2782690cdf8b admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:10.071665 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac601a2a-1c20-4964-9238-2782690cdf8b admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:10.078108 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfccaac-b3e0-4616-853d-de3f14747e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:10.078890 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfccaac-b3e0-4616-853d-de3f14747e34 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:10.079316 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfccaac-b3e0-4616-853d-de3f14747e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:10.079717 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfccaac-b3e0-4616-853d-de3f14747e34 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:10.080529 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfccaac-b3e0-4616-853d-de3f14747e34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:10.082318 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfccaac-b3e0-4616-853d-de3f14747e34 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:10.096027 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1169/2344] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:51:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:51:10.100604 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04981e62-6b2a-4799-89be-3fd1de3dc84c admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:10.122059 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04981e62-6b2a-4799-89be-3fd1de3dc84c admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:10.130417 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0fa741-ef41-4172-9d68-cb7df88c3d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:10.132899 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0fa741-ef41-4172-9d68-cb7df88c3d64 admin admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:10.134947 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0fa741-ef41-4172-9d68-cb7df88c3d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:10.135476 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0fa741-ef41-4172-9d68-cb7df88c3d64 admin admin] PATH_INFO: `/v3/projects/3a05bbed64d3458c8ddcc28532a23b44/users/8053f6174866478e8434907432f55085/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:10.155395 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0fa741-ef41-4172-9d68-cb7df88c3d64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a05bbed64d3458c8ddcc28532a23b44, user_id=8053f6174866478e8434907432f55085, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:10.158666 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0fa741-ef41-4172-9d68-cb7df88c3d64 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:10.193352 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1176/2345] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:51:10 2022] PUT /identity/v3/projects/3a05bbed64d3458c8ddcc28532a23b44/users/8053f6174866478e8434907432f55085/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:51:10.205762 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97bac75-4678-4a8c-b601-008b5fc7d069 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:10.206246 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97bac75-4678-4a8c-b601-008b5fc7d069 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:10.206605 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97bac75-4678-4a8c-b601-008b5fc7d069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:10.245841 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-c97bac75-4678-4a8c-b601-008b5fc7d069 None None] MFA Rules not processed for user `8053f6174866478e8434907432f55085`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:51:10.247472 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-c97bac75-4678-4a8c-b601-008b5fc7d069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:10.347396 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1170/2346] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:51:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:51:10.348628 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0fa741-ef41-4172-9d68-cb7df88c3d64 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:10.428757 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0fa741-ef41-4172-9d68-cb7df88c3d64 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:10.433979 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce65318-d15d-4013-af37-5f16f7cc0149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:10.434682 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce65318-d15d-4013-af37-5f16f7cc0149 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:10.435031 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce65318-d15d-4013-af37-5f16f7cc0149 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:10.435896 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce65318-d15d-4013-af37-5f16f7cc0149 admin admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:10.436566 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce65318-d15d-4013-af37-5f16f7cc0149 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:10.439450 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce65318-d15d-4013-af37-5f16f7cc0149 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:10.495285 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1177/2347] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:51:10 2022] POST /identity/v3/projects => generated 383 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:51:10.501545 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97bac75-4678-4a8c-b601-008b5fc7d069 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:10.526478 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97bac75-4678-4a8c-b601-008b5fc7d069 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:10.542882 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:51:10.611801 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eec6da-467d-4237-bcb2-2e7390fe8e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:10.612619 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eec6da-467d-4237-bcb2-2e7390fe8e0d admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:10.613116 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eec6da-467d-4237-bcb2-2e7390fe8e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:10.613444 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eec6da-467d-4237-bcb2-2e7390fe8e0d admin admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:10.614396 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eec6da-467d-4237-bcb2-2e7390fe8e0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:10.616573 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eec6da-467d-4237-bcb2-2e7390fe8e0d admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:10.638922 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1171/2348] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:51:10 2022] POST /identity/v3/users => generated 298 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:51:10.642941 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce65318-d15d-4013-af37-5f16f7cc0149 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:10.653712 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce65318-d15d-4013-af37-5f16f7cc0149 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:10.659599 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400b62e4-0521-4426-9df0-bc0635984e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:10.660245 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400b62e4-0521-4426-9df0-bc0635984e7e admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:10.660533 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400b62e4-0521-4426-9df0-bc0635984e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:10.660808 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400b62e4-0521-4426-9df0-bc0635984e7e admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:10.661445 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400b62e4-0521-4426-9df0-bc0635984e7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:10.663193 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400b62e4-0521-4426-9df0-bc0635984e7e admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:10.664809 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:51:10.676930 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1178/2349] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:51:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:51:10.681011 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eec6da-467d-4237-bcb2-2e7390fe8e0d admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:10.692101 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eec6da-467d-4237-bcb2-2e7390fe8e0d admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:10.697314 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640cc6e6-e070-4c34-b169-3e54e83c350b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:10.697966 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640cc6e6-e070-4c34-b169-3e54e83c350b admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:10.698246 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640cc6e6-e070-4c34-b169-3e54e83c350b admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:10.698523 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640cc6e6-e070-4c34-b169-3e54e83c350b admin admin] PATH_INFO: `/v3/projects/6ee14e6c9ee5481480f407640b4fbe9f/users/c1883a08282b45d9aa902bf59478f93a/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:10.714448 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640cc6e6-e070-4c34-b169-3e54e83c350b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ee14e6c9ee5481480f407640b4fbe9f, user_id=c1883a08282b45d9aa902bf59478f93a, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:10.717214 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640cc6e6-e070-4c34-b169-3e54e83c350b admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:10.730630 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1172/2350] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:51:10 2022] PUT /identity/v3/projects/6ee14e6c9ee5481480f407640b4fbe9f/users/c1883a08282b45d9aa902bf59478f93a/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:51:10.734893 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400b62e4-0521-4426-9df0-bc0635984e7e admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:10.745079 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400b62e4-0521-4426-9df0-bc0635984e7e admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:10.807007 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f6502-e7e1-4442-86a4-3063a00ebc05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:10.807653 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f6502-e7e1-4442-86a4-3063a00ebc05 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:10.807928 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f6502-e7e1-4442-86a4-3063a00ebc05 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:10.808195 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f6502-e7e1-4442-86a4-3063a00ebc05 admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:10.808850 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868f6502-e7e1-4442-86a4-3063a00ebc05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:10.811087 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868f6502-e7e1-4442-86a4-3063a00ebc05 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:10.821025 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1179/2351] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:51:10 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:51:10.825284 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640cc6e6-e070-4c34-b169-3e54e83c350b admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:10.836569 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640cc6e6-e070-4c34-b169-3e54e83c350b admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:10.841110 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f67f57-a8b0-4b5c-a942-ec0325f6f565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:10.841873 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f67f57-a8b0-4b5c-a942-ec0325f6f565 admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:10.842153 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f67f57-a8b0-4b5c-a942-ec0325f6f565 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:10.842424 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f67f57-a8b0-4b5c-a942-ec0325f6f565 admin admin] PATH_INFO: `/v3/projects/6ee14e6c9ee5481480f407640b4fbe9f/users/c1883a08282b45d9aa902bf59478f93a/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:10.844383 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f67f57-a8b0-4b5c-a942-ec0325f6f565 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ee14e6c9ee5481480f407640b4fbe9f, user_id=c1883a08282b45d9aa902bf59478f93a, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:10.847031 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f67f57-a8b0-4b5c-a942-ec0325f6f565 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:10.857057 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1173/2352] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:51:10 2022] PUT /identity/v3/projects/6ee14e6c9ee5481480f407640b4fbe9f/users/c1883a08282b45d9aa902bf59478f93a/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:51:10.868164 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18adc13b-575f-47cd-a364-8f9d9b40bc99 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:10.868448 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18adc13b-575f-47cd-a364-8f9d9b40bc99 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:10.868716 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18adc13b-575f-47cd-a364-8f9d9b40bc99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:10.898592 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-18adc13b-575f-47cd-a364-8f9d9b40bc99 None None] MFA Rules not processed for user `c1883a08282b45d9aa902bf59478f93a`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:51:10.900106 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-18adc13b-575f-47cd-a364-8f9d9b40bc99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:10.971912 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1180/2353] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:51:10 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:51:10.983825 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f67f57-a8b0-4b5c-a942-ec0325f6f565 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:10.985828 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-90f67f57-a8b0-4b5c-a942-ec0325f6f565 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:11.050162 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f67f57-a8b0-4b5c-a942-ec0325f6f565 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:11.055030 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d689cee2-7d4c-455f-a6da-72534c5a1c5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:11.055709 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d689cee2-7d4c-455f-a6da-72534c5a1c5c service cinder] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:11.055995 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d689cee2-7d4c-455f-a6da-72534c5a1c5c service cinder] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:11.056265 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d689cee2-7d4c-455f-a6da-72534c5a1c5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:11.066163 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d689cee2-7d4c-455f-a6da-72534c5a1c5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:11.067640 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d689cee2-7d4c-455f-a6da-72534c5a1c5c service cinder] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:11.077614 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1174/2354] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:51:10 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:51:11.624197 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18adc13b-575f-47cd-a364-8f9d9b40bc99 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:11.635182 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18adc13b-575f-47cd-a364-8f9d9b40bc99 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:11.640077 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07959588-829b-4aa2-ba48-088d3b7c4603 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:11.640692 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07959588-829b-4aa2-ba48-088d3b7c4603 service cinder] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:11.640969 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07959588-829b-4aa2-ba48-088d3b7c4603 service cinder] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:11.641239 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07959588-829b-4aa2-ba48-088d3b7c4603 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:11.709767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07959588-829b-4aa2-ba48-088d3b7c4603 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:11.711362 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07959588-829b-4aa2-ba48-088d3b7c4603 service cinder] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:11.720685 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1181/2355] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:51:11 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:51:12.073555 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec1562-bc94-4bf5-be7a-04da269196c3 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:12.074366 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec1562-bc94-4bf5-be7a-04da269196c3 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:12.074833 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec1562-bc94-4bf5-be7a-04da269196c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:12.110965 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-c1ec1562-bc94-4bf5-be7a-04da269196c3 None None] MFA Rules not processed for user `bb44e7aef0c7436d93a7233307b5f095`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:51:12.112898 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-c1ec1562-bc94-4bf5-be7a-04da269196c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:12.201531 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1175/2356] 10.222.0.15 () {58 vars in 967 bytes} [Sun Sep 11 10:51:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:51:12.217339 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07959588-829b-4aa2-ba48-088d3b7c4603 service cinder] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:12.219149 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-07959588-829b-4aa2-ba48-088d3b7c4603 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:12.239945 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:51:12.303798 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07959588-829b-4aa2-ba48-088d3b7c4603 service cinder] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:12.315815 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18936aae-5a1f-43a4-9779-601310a1810a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7ca566e66a45b29e3d02f47bf84981', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:12.316528 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18936aae-5a1f-43a4-9779-601310a1810a service swift] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:12.316884 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18936aae-5a1f-43a4-9779-601310a1810a service swift] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:12.317228 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18936aae-5a1f-43a4-9779-601310a1810a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:12.327773 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18936aae-5a1f-43a4-9779-601310a1810a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:12.329678 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18936aae-5a1f-43a4-9779-601310a1810a service swift] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:12.338772 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1182/2357] 10.222.0.15 () {60 vars in 1384 bytes} [Sun Sep 11 10:51:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 10:51:14.346655 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75696942-6dd0-4784-8cce-0c34641fa8af None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:14.347188 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75696942-6dd0-4784-8cce-0c34641fa8af None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:14.347188 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75696942-6dd0-4784-8cce-0c34641fa8af None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:14.349155 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1176/2358] 10.222.0.15 () {54 vars in 921 bytes} [Sun Sep 11 10:51:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:51:14.377391 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596c61a9-a514-4f6e-9ea3-ba68e28a29f4 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:14.377868 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596c61a9-a514-4f6e-9ea3-ba68e28a29f4 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:14.378226 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596c61a9-a514-4f6e-9ea3-ba68e28a29f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:14.399939 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-596c61a9-a514-4f6e-9ea3-ba68e28a29f4 None None] MFA Rules not processed for user `08477b4c88f04c9e97391aba401879e0`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:51:14.401155 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-596c61a9-a514-4f6e-9ea3-ba68e28a29f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:14.421245 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75696942-6dd0-4784-8cce-0c34641fa8af None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:14.423478 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-75696942-6dd0-4784-8cce-0c34641fa8af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:14.454220 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:51:14.480070 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1183/2359] 10.222.0.15 () {58 vars in 1018 bytes} [Sun Sep 11 10:51:14 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:51:14.493657 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a91793-c041-473a-95a2-f460524a9198 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:14.494066 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a91793-c041-473a-95a2-f460524a9198 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:14.494355 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a91793-c041-473a-95a2-f460524a9198 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:14.496008 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1184/2360] 10.222.0.15 () {54 vars in 961 bytes} [Sun Sep 11 10:51:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:51:14.500814 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2440357-10fa-4ff3-9d46-9095f4d8c57d None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:14.501111 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2440357-10fa-4ff3-9d46-9095f4d8c57d None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:14.501410 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2440357-10fa-4ff3-9d46-9095f4d8c57d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:14.517157 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75696942-6dd0-4784-8cce-0c34641fa8af None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:14.521684 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ccc2ea-909d-4bbf-b60a-122a4e8b4cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:14.522447 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ccc2ea-909d-4bbf-b60a-122a4e8b4cbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:14.522568 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-a2440357-10fa-4ff3-9d46-9095f4d8c57d None None] MFA Rules not processed for user `08477b4c88f04c9e97391aba401879e0`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:51:14.522730 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ccc2ea-909d-4bbf-b60a-122a4e8b4cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:14.523000 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ccc2ea-909d-4bbf-b60a-122a4e8b4cbe admin admin] PATH_INFO: `/v3/users/30f5e5a2ce8749d692600f1d2ab4fdb9` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:14.524062 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-a2440357-10fa-4ff3-9d46-9095f4d8c57d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:14.524202 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ccc2ea-909d-4bbf-b60a-122a4e8b4cbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=30f5e5a2ce8749d692600f1d2ab4fdb9)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:14.526071 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ccc2ea-909d-4bbf-b60a-122a4e8b4cbe admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:14.536237 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1185/2361] 10.222.0.15 () {58 vars in 1058 bytes} [Sun Sep 11 10:51:14 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 10:51:14.543835 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2440357-10fa-4ff3-9d46-9095f4d8c57d None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:14.554548 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2440357-10fa-4ff3-9d46-9095f4d8c57d None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:14.561530 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c293b3a6-4401-4423-b881-175b94d50f59 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08477b4c88f04c9e97391aba401879e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:14.562233 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293b3a6-4401-4423-b881-175b94d50f59 service placement] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:14.562622 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293b3a6-4401-4423-b881-175b94d50f59 service placement] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:14.562920 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293b3a6-4401-4423-b881-175b94d50f59 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:14.572724 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c293b3a6-4401-4423-b881-175b94d50f59 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:14.574445 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c293b3a6-4401-4423-b881-175b94d50f59 service placement] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:14.585642 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1186/2362] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:51:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 10:51:14.585823 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-59ccc2ea-909d-4bbf-b60a-122a4e8b4cbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30f5e5a2ce8749d692600f1d2ab4fdb9'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:14.594249 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-59ccc2ea-909d-4bbf-b60a-122a4e8b4cbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30f5e5a2ce8749d692600f1d2ab4fdb9'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:14.594979 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-59ccc2ea-909d-4bbf-b60a-122a4e8b4cbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30f5e5a2ce8749d692600f1d2ab4fdb9'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:14.595716 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-59ccc2ea-909d-4bbf-b60a-122a4e8b4cbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30f5e5a2ce8749d692600f1d2ab4fdb9'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:14.610433 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-59ccc2ea-909d-4bbf-b60a-122a4e8b4cbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30f5e5a2ce8749d692600f1d2ab4fdb9'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:14.632168 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1177/2363] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:51:14 2022] DELETE /identity/v3/users/30f5e5a2ce8749d692600f1d2ab4fdb9 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 10:51:14.820622 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c293b3a6-4401-4423-b881-175b94d50f59 service placement] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:14.832457 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c293b3a6-4401-4423-b881-175b94d50f59 service placement] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:14.893012 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7ae8fc-eaf0-4aa8-830e-7c352265a24e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:14.893653 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7ae8fc-eaf0-4aa8-830e-7c352265a24e admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:14.893933 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7ae8fc-eaf0-4aa8-830e-7c352265a24e admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:14.894208 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7ae8fc-eaf0-4aa8-830e-7c352265a24e admin admin] PATH_INFO: `/v3/projects/6e5993a0a4964d7597bf4e2e8f252171` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:14.895369 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7ae8fc-eaf0-4aa8-830e-7c352265a24e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e5993a0a4964d7597bf4e2e8f252171)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:14.897301 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7ae8fc-eaf0-4aa8-830e-7c352265a24e admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:14.923183 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:51:14.946805 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-db7ae8fc-eaf0-4aa8-830e-7c352265a24e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e5993a0a4964d7597bf4e2e8f252171', 'request_id': 'req-db7ae8fc-eaf0-4aa8-830e-7c352265a24e'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:14.953368 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-db7ae8fc-eaf0-4aa8-830e-7c352265a24e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e5993a0a4964d7597bf4e2e8f252171', 'request_id': 'req-db7ae8fc-eaf0-4aa8-830e-7c352265a24e'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:14.959238 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-db7ae8fc-eaf0-4aa8-830e-7c352265a24e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e5993a0a4964d7597bf4e2e8f252171', 'request_id': 'req-db7ae8fc-eaf0-4aa8-830e-7c352265a24e'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:14.980353 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-db7ae8fc-eaf0-4aa8-830e-7c352265a24e admin admin] The token cache is being invalidate because project 6e5993a0a4964d7597bf4e2e8f252171 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:51:14.980679 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-db7ae8fc-eaf0-4aa8-830e-7c352265a24e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:14.981306 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-db7ae8fc-eaf0-4aa8-830e-7c352265a24e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:14.983791 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1187/2364] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:51:14 2022] DELETE /identity/v3/projects/6e5993a0a4964d7597bf4e2e8f252171 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:51:14.989107 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56095342-9566-402b-9a70-3a5fb4a500b6 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:14.989393 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56095342-9566-402b-9a70-3a5fb4a500b6 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:14.989666 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56095342-9566-402b-9a70-3a5fb4a500b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:15.010519 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-56095342-9566-402b-9a70-3a5fb4a500b6 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:51:15.011827 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-56095342-9566-402b-9a70-3a5fb4a500b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:15.072531 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1178/2365] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:51:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:51:15.080910 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c623fede-8a61-488f-8326-ef1f21525595 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:15.081284 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c623fede-8a61-488f-8326-ef1f21525595 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:15.081638 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c623fede-8a61-488f-8326-ef1f21525595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:15.102075 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-c623fede-8a61-488f-8326-ef1f21525595 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:51:15.103882 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-c623fede-8a61-488f-8326-ef1f21525595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:15.112282 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1188/2366] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:51:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:51:15.116556 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56095342-9566-402b-9a70-3a5fb4a500b6 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:15.127828 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56095342-9566-402b-9a70-3a5fb4a500b6 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:15.132444 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6bd3a7-3793-4f5f-a44d-6dd991b117c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:15.133042 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6bd3a7-3793-4f5f-a44d-6dd991b117c0 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:15.133320 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6bd3a7-3793-4f5f-a44d-6dd991b117c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:15.133586 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6bd3a7-3793-4f5f-a44d-6dd991b117c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:15.134253 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6bd3a7-3793-4f5f-a44d-6dd991b117c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:51:15.134547 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6bd3a7-3793-4f5f-a44d-6dd991b117c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:15.136174 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6bd3a7-3793-4f5f-a44d-6dd991b117c0 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:15.148823 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1179/2367] 10.222.0.15 () {58 vars in 1153 bytes} [Sun Sep 11 10:51:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:51:15.152883 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c623fede-8a61-488f-8326-ef1f21525595 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:15.163090 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c623fede-8a61-488f-8326-ef1f21525595 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:15.167908 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561fffa8-54db-4096-a017-777437f9ac08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:15.168502 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561fffa8-54db-4096-a017-777437f9ac08 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:15.168780 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561fffa8-54db-4096-a017-777437f9ac08 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:15.169048 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561fffa8-54db-4096-a017-777437f9ac08 admin admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:15.169733 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561fffa8-54db-4096-a017-777437f9ac08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:15.172015 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561fffa8-54db-4096-a017-777437f9ac08 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:15.203522 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1189/2368] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:51:15 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 10:51:15.207550 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6bd3a7-3793-4f5f-a44d-6dd991b117c0 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:15.219014 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6bd3a7-3793-4f5f-a44d-6dd991b117c0 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:15.239397 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:51:15.283704 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104aa43b-bca7-4b63-8e62-dab8351a86bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:15.288200 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104aa43b-bca7-4b63-8e62-dab8351a86bf admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:15.288706 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104aa43b-bca7-4b63-8e62-dab8351a86bf admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:15.289050 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104aa43b-bca7-4b63-8e62-dab8351a86bf admin admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:15.289731 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104aa43b-bca7-4b63-8e62-dab8351a86bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:15.291840 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104aa43b-bca7-4b63-8e62-dab8351a86bf admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:15.317029 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1180/2369] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:51:15 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 10:51:15.321021 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561fffa8-54db-4096-a017-777437f9ac08 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:15.331402 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561fffa8-54db-4096-a017-777437f9ac08 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:15.336195 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c17b3a8-5e09-4ea2-9c54-bdad99e2da93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:15.336801 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c17b3a8-5e09-4ea2-9c54-bdad99e2da93 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:15.337108 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c17b3a8-5e09-4ea2-9c54-bdad99e2da93 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:15.337365 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c17b3a8-5e09-4ea2-9c54-bdad99e2da93 admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:15.338001 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c17b3a8-5e09-4ea2-9c54-bdad99e2da93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:15.339636 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c17b3a8-5e09-4ea2-9c54-bdad99e2da93 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:15.350108 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1190/2370] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:51:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:51:15.356700 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104aa43b-bca7-4b63-8e62-dab8351a86bf admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:15.369614 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104aa43b-bca7-4b63-8e62-dab8351a86bf admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:15.374658 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d1e6ba-ab15-4587-997b-66f6b4ae88ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:15.375262 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d1e6ba-ab15-4587-997b-66f6b4ae88ef admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:15.375582 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d1e6ba-ab15-4587-997b-66f6b4ae88ef admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:15.375856 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d1e6ba-ab15-4587-997b-66f6b4ae88ef admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:15.376489 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d1e6ba-ab15-4587-997b-66f6b4ae88ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:15.378081 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d1e6ba-ab15-4587-997b-66f6b4ae88ef admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:15.389138 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1181/2371] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:51:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:51:15.393291 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c17b3a8-5e09-4ea2-9c54-bdad99e2da93 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:15.403132 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c17b3a8-5e09-4ea2-9c54-bdad99e2da93 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:15.408989 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d076eb-3e97-49a0-aa94-063309692201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:15.409649 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d076eb-3e97-49a0-aa94-063309692201 admin admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:15.409927 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d076eb-3e97-49a0-aa94-063309692201 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:15.410203 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d076eb-3e97-49a0-aa94-063309692201 admin admin] PATH_INFO: `/v3/projects/73cbd4a12d444b0ba5d4c37d12f2ea49/users/8902fe9b4ce54ad398422722101a947a/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:15.426154 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d076eb-3e97-49a0-aa94-063309692201 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73cbd4a12d444b0ba5d4c37d12f2ea49, user_id=8902fe9b4ce54ad398422722101a947a, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:15.428860 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d076eb-3e97-49a0-aa94-063309692201 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:15.441978 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1191/2372] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:51:15 2022] PUT /identity/v3/projects/73cbd4a12d444b0ba5d4c37d12f2ea49/users/8902fe9b4ce54ad398422722101a947a/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:51:15.446944 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d6fe48-b9c6-4996-8c05-5713c718016d None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:15.447256 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d6fe48-b9c6-4996-8c05-5713c718016d None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:15.447567 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d6fe48-b9c6-4996-8c05-5713c718016d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:15.467286 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-c6d6fe48-b9c6-4996-8c05-5713c718016d None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:51:15.468525 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-c6d6fe48-b9c6-4996-8c05-5713c718016d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:15.528337 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1182/2373] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:51:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:51:15.538376 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d076eb-3e97-49a0-aa94-063309692201 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:15.540250 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-23d076eb-3e97-49a0-aa94-063309692201 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:15.597539 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d076eb-3e97-49a0-aa94-063309692201 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:15.603007 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fd0517-fe2e-4226-8a14-0f366042f419 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:15.604085 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fd0517-fe2e-4226-8a14-0f366042f419 service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:15.604767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fd0517-fe2e-4226-8a14-0f366042f419 service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:15.605269 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fd0517-fe2e-4226-8a14-0f366042f419 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:15.616887 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fd0517-fe2e-4226-8a14-0f366042f419 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:15.618259 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fd0517-fe2e-4226-8a14-0f366042f419 service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:15.628645 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1192/2374] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:51:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:51:15.952787 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d6fe48-b9c6-4996-8c05-5713c718016d None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:15.954286 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-c6d6fe48-b9c6-4996-8c05-5713c718016d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:16.016641 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d6fe48-b9c6-4996-8c05-5713c718016d None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:16.021633 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74999ee-453a-4d75-8436-adcffb1f7218 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:16.022246 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74999ee-453a-4d75-8436-adcffb1f7218 service cinder] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:16.022523 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74999ee-453a-4d75-8436-adcffb1f7218 service cinder] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:16.022797 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74999ee-453a-4d75-8436-adcffb1f7218 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:16.024557 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-e74999ee-453a-4d75-8436-adcffb1f7218 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:16.087286 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:51:16.089706 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74999ee-453a-4d75-8436-adcffb1f7218 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:16.091152 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74999ee-453a-4d75-8436-adcffb1f7218 service cinder] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:16.102079 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1183/2375] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:51:15 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 10:51:18.488355 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fd0517-fe2e-4226-8a14-0f366042f419 service neutron] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:18.491683 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-84fd0517-fe2e-4226-8a14-0f366042f419 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:18.509247 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fd0517-fe2e-4226-8a14-0f366042f419 service neutron] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:18.514437 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fc83cb-cbae-4784-b6cb-2e0301f38361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:18.515141 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc83cb-cbae-4784-b6cb-2e0301f38361 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:18.515627 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc83cb-cbae-4784-b6cb-2e0301f38361 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:18.515909 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc83cb-cbae-4784-b6cb-2e0301f38361 admin admin] PATH_INFO: `/v3/users/2d965d07755d4590accea12dbf84c46b` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:18.517065 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fc83cb-cbae-4784-b6cb-2e0301f38361 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d965d07755d4590accea12dbf84c46b)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:18.519204 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fc83cb-cbae-4784-b6cb-2e0301f38361 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:18.576585 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-58fc83cb-cbae-4784-b6cb-2e0301f38361 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d965d07755d4590accea12dbf84c46b'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:18.578819 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:51:18.586712 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-58fc83cb-cbae-4784-b6cb-2e0301f38361 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d965d07755d4590accea12dbf84c46b'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:18.587859 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-58fc83cb-cbae-4784-b6cb-2e0301f38361 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d965d07755d4590accea12dbf84c46b'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:18.588752 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-58fc83cb-cbae-4784-b6cb-2e0301f38361 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d965d07755d4590accea12dbf84c46b'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:18.606008 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-58fc83cb-cbae-4784-b6cb-2e0301f38361 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d965d07755d4590accea12dbf84c46b'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:18.621172 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1193/2376] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:51:18 2022] DELETE /identity/v3/users/2d965d07755d4590accea12dbf84c46b => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:51:18.626205 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe71b7-fa8b-47d1-8b41-2ac092e38d71 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:18.627308 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe71b7-fa8b-47d1-8b41-2ac092e38d71 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:18.627757 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe71b7-fa8b-47d1-8b41-2ac092e38d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:18.653554 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-7fbe71b7-fa8b-47d1-8b41-2ac092e38d71 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:51:18.655975 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-7fbe71b7-fa8b-47d1-8b41-2ac092e38d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:18.730057 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1184/2377] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:51:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 10:51:18.739542 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fc83cb-cbae-4784-b6cb-2e0301f38361 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:18.741047 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-58fc83cb-cbae-4784-b6cb-2e0301f38361 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:18.816010 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fc83cb-cbae-4784-b6cb-2e0301f38361 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:18.821852 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8701986b-fe0a-4294-81cd-8d5173f1d905 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:18.822792 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8701986b-fe0a-4294-81cd-8d5173f1d905 service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:18.822941 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8701986b-fe0a-4294-81cd-8d5173f1d905 service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:18.823123 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8701986b-fe0a-4294-81cd-8d5173f1d905 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:18.833628 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8701986b-fe0a-4294-81cd-8d5173f1d905 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:18.835075 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8701986b-fe0a-4294-81cd-8d5173f1d905 service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:18.845991 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1194/2378] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:51:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 10:51:19.454742 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbe71b7-fa8b-47d1-8b41-2ac092e38d71 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:19.457336 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-7fbe71b7-fa8b-47d1-8b41-2ac092e38d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:19.476012 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbe71b7-fa8b-47d1-8b41-2ac092e38d71 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:19.483549 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de490ad5-be37-4c21-ac20-fef427e052d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:19.484268 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de490ad5-be37-4c21-ac20-fef427e052d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:19.484543 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de490ad5-be37-4c21-ac20-fef427e052d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:19.484818 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de490ad5-be37-4c21-ac20-fef427e052d0 admin admin] PATH_INFO: `/v3/projects/155e70fb43d34b6d8cdfd16b7a318057` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:19.486036 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de490ad5-be37-4c21-ac20-fef427e052d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=155e70fb43d34b6d8cdfd16b7a318057)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:19.488243 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de490ad5-be37-4c21-ac20-fef427e052d0 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:19.551981 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-de490ad5-be37-4c21-ac20-fef427e052d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '155e70fb43d34b6d8cdfd16b7a318057', 'request_id': 'req-de490ad5-be37-4c21-ac20-fef427e052d0'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:19.559164 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-de490ad5-be37-4c21-ac20-fef427e052d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '155e70fb43d34b6d8cdfd16b7a318057', 'request_id': 'req-de490ad5-be37-4c21-ac20-fef427e052d0'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:19.571163 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-de490ad5-be37-4c21-ac20-fef427e052d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '155e70fb43d34b6d8cdfd16b7a318057', 'request_id': 'req-de490ad5-be37-4c21-ac20-fef427e052d0'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:19.583745 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-de490ad5-be37-4c21-ac20-fef427e052d0 admin admin] The token cache is being invalidate because project 155e70fb43d34b6d8cdfd16b7a318057 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90653) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:51:19.584037 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-de490ad5-be37-4c21-ac20-fef427e052d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:19.584715 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-de490ad5-be37-4c21-ac20-fef427e052d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:19.589622 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1185/2379] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:51:19 2022] DELETE /identity/v3/projects/155e70fb43d34b6d8cdfd16b7a318057 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:51:19.590893 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8701986b-fe0a-4294-81cd-8d5173f1d905 service neutron] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:19.601481 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8701986b-fe0a-4294-81cd-8d5173f1d905 service neutron] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:19.671947 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f06c01-520d-416b-b0f3-92b646f687da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:19.672895 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f06c01-520d-416b-b0f3-92b646f687da admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:19.673210 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f06c01-520d-416b-b0f3-92b646f687da admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:19.673720 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f06c01-520d-416b-b0f3-92b646f687da admin admin] PATH_INFO: `/v3/users/8053f6174866478e8434907432f55085` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:19.676149 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f06c01-520d-416b-b0f3-92b646f687da admin admin] RBAC: Authorizing `identity:delete_user(user_id=8053f6174866478e8434907432f55085)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:19.678574 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f06c01-520d-416b-b0f3-92b646f687da admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:19.754433 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-20f06c01-520d-416b-b0f3-92b646f687da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8053f6174866478e8434907432f55085'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:19.766550 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-20f06c01-520d-416b-b0f3-92b646f687da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8053f6174866478e8434907432f55085'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:19.767181 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-20f06c01-520d-416b-b0f3-92b646f687da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8053f6174866478e8434907432f55085'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:19.767811 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-20f06c01-520d-416b-b0f3-92b646f687da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8053f6174866478e8434907432f55085'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:19.784300 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:51:19.787086 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-20f06c01-520d-416b-b0f3-92b646f687da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8053f6174866478e8434907432f55085'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:19.811576 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1195/2380] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:51:19 2022] DELETE /identity/v3/users/8053f6174866478e8434907432f55085 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 10:51:20.523778 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de490ad5-be37-4c21-ac20-fef427e052d0 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:20.525300 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-de490ad5-be37-4c21-ac20-fef427e052d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:20.540463 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de490ad5-be37-4c21-ac20-fef427e052d0 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:20.542316 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ec5c68-80ee-4a03-a977-d1d7baf7dc5a None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:20.542843 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ec5c68-80ee-4a03-a977-d1d7baf7dc5a None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:20.542843 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ec5c68-80ee-4a03-a977-d1d7baf7dc5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:20.600990 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-e4ec5c68-80ee-4a03-a977-d1d7baf7dc5a None None] MFA Rules not processed for user `8902fe9b4ce54ad398422722101a947a`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:51:20.602421 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-e4ec5c68-80ee-4a03-a977-d1d7baf7dc5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:20.631029 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526c3054-c765-4d04-b7a4-983051f5c9d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:20.631731 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526c3054-c765-4d04-b7a4-983051f5c9d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:20.632011 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526c3054-c765-4d04-b7a4-983051f5c9d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:20.632282 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526c3054-c765-4d04-b7a4-983051f5c9d5 admin admin] PATH_INFO: `/v3/projects/3a05bbed64d3458c8ddcc28532a23b44` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:20.633613 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526c3054-c765-4d04-b7a4-983051f5c9d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a05bbed64d3458c8ddcc28532a23b44)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:20.635683 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526c3054-c765-4d04-b7a4-983051f5c9d5 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:20.676211 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:51:20.694530 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1196/2381] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:51:20 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 10:51:20.698986 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ec5c68-80ee-4a03-a977-d1d7baf7dc5a None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:20.700497 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-e4ec5c68-80ee-4a03-a977-d1d7baf7dc5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:20.706899 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-526c3054-c765-4d04-b7a4-983051f5c9d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a05bbed64d3458c8ddcc28532a23b44', 'request_id': 'req-526c3054-c765-4d04-b7a4-983051f5c9d5'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:20.713490 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-526c3054-c765-4d04-b7a4-983051f5c9d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a05bbed64d3458c8ddcc28532a23b44', 'request_id': 'req-526c3054-c765-4d04-b7a4-983051f5c9d5'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:20.714485 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ec5c68-80ee-4a03-a977-d1d7baf7dc5a None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:20.723097 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-526c3054-c765-4d04-b7a4-983051f5c9d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a05bbed64d3458c8ddcc28532a23b44', 'request_id': 'req-526c3054-c765-4d04-b7a4-983051f5c9d5'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:20.735414 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-526c3054-c765-4d04-b7a4-983051f5c9d5 admin admin] The token cache is being invalidate because project 3a05bbed64d3458c8ddcc28532a23b44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90653) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:51:20.735741 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-526c3054-c765-4d04-b7a4-983051f5c9d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:20.736393 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-526c3054-c765-4d04-b7a4-983051f5c9d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:20.737970 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1186/2382] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:51:20 2022] DELETE /identity/v3/projects/3a05bbed64d3458c8ddcc28532a23b44 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:51:20.743226 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526c3054-c765-4d04-b7a4-983051f5c9d5 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:20.744735 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-526c3054-c765-4d04-b7a4-983051f5c9d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:20.779094 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b9b3d9-047c-4605-b1e7-1aa1357273b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:20.779953 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b9b3d9-047c-4605-b1e7-1aa1357273b2 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:20.780235 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b9b3d9-047c-4605-b1e7-1aa1357273b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:20.780583 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b9b3d9-047c-4605-b1e7-1aa1357273b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:20.781359 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b9b3d9-047c-4605-b1e7-1aa1357273b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:20.783156 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b9b3d9-047c-4605-b1e7-1aa1357273b2 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:20.815716 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1197/2383] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:51:20 2022] POST /identity/v3/projects => generated 383 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 10:51:20.816681 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b9b3d9-047c-4605-b1e7-1aa1357273b2 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:20.818263 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-c9b9b3d9-047c-4605-b1e7-1aa1357273b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:20.822020 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526c3054-c765-4d04-b7a4-983051f5c9d5 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:20.882417 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d542f5f-ad89-48ee-99be-bb3dfcba185a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:20.883060 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d542f5f-ad89-48ee-99be-bb3dfcba185a admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:20.883348 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d542f5f-ad89-48ee-99be-bb3dfcba185a admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:20.883653 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d542f5f-ad89-48ee-99be-bb3dfcba185a admin admin] PATH_INFO: `/v3/users/c1883a08282b45d9aa902bf59478f93a` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:20.884902 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d542f5f-ad89-48ee-99be-bb3dfcba185a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1883a08282b45d9aa902bf59478f93a)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:20.886876 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d542f5f-ad89-48ee-99be-bb3dfcba185a admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:20.890002 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b9b3d9-047c-4605-b1e7-1aa1357273b2 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:20.894693 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dfcd5c-1f81-4398-a02e-fe96c0f722c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:20.895298 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dfcd5c-1f81-4398-a02e-fe96c0f722c4 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:20.895626 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dfcd5c-1f81-4398-a02e-fe96c0f722c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:20.895900 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dfcd5c-1f81-4398-a02e-fe96c0f722c4 admin admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:20.896580 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dfcd5c-1f81-4398-a02e-fe96c0f722c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:20.898496 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dfcd5c-1f81-4398-a02e-fe96c0f722c4 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:20.907373 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:51:20.927007 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1198/2384] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:51:20 2022] POST /identity/v3/users => generated 298 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 10:51:20.928201 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dfcd5c-1f81-4398-a02e-fe96c0f722c4 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:20.937846 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-8d542f5f-ad89-48ee-99be-bb3dfcba185a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1883a08282b45d9aa902bf59478f93a'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:20.938498 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dfcd5c-1f81-4398-a02e-fe96c0f722c4 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:20.944596 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5dc210-0b6f-4c4f-8063-8328e697e23f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:20.945448 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5dc210-0b6f-4c4f-8063-8328e697e23f admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:20.945573 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-8d542f5f-ad89-48ee-99be-bb3dfcba185a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1883a08282b45d9aa902bf59478f93a'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:20.946095 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5dc210-0b6f-4c4f-8063-8328e697e23f admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:20.946386 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-8d542f5f-ad89-48ee-99be-bb3dfcba185a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1883a08282b45d9aa902bf59478f93a'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:20.946540 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5dc210-0b6f-4c4f-8063-8328e697e23f admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:20.947055 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-8d542f5f-ad89-48ee-99be-bb3dfcba185a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1883a08282b45d9aa902bf59478f93a'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:20.947231 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5dc210-0b6f-4c4f-8063-8328e697e23f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:20.949432 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5dc210-0b6f-4c4f-8063-8328e697e23f admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:20.952021 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:51:20.955858 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-8d542f5f-ad89-48ee-99be-bb3dfcba185a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1883a08282b45d9aa902bf59478f93a'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:20.963029 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1199/2385] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:51:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:51:20.964446 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5dc210-0b6f-4c4f-8063-8328e697e23f admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:20.966494 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-5d5dc210-0b6f-4c4f-8063-8328e697e23f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:20.975913 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1187/2386] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:51:20 2022] DELETE /identity/v3/users/c1883a08282b45d9aa902bf59478f93a => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:51:20.983011 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5dc210-0b6f-4c4f-8063-8328e697e23f admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:21.046629 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf09c6ae-378a-421f-b67d-fd47b3c8b1b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:21.047295 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf09c6ae-378a-421f-b67d-fd47b3c8b1b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:21.047745 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf09c6ae-378a-421f-b67d-fd47b3c8b1b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:21.048023 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf09c6ae-378a-421f-b67d-fd47b3c8b1b6 admin admin] PATH_INFO: `/v3/projects/47c13d1196e0434cbb5e0b1469625e87/users/55afe027d2b046e094eef686ccbbf5e4/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:21.063121 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf09c6ae-378a-421f-b67d-fd47b3c8b1b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47c13d1196e0434cbb5e0b1469625e87, user_id=55afe027d2b046e094eef686ccbbf5e4, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:21.066075 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf09c6ae-378a-421f-b67d-fd47b3c8b1b6 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:21.083208 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1200/2387] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:51:20 2022] PUT /identity/v3/projects/47c13d1196e0434cbb5e0b1469625e87/users/55afe027d2b046e094eef686ccbbf5e4/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:51:21.085562 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d542f5f-ad89-48ee-99be-bb3dfcba185a admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:21.097499 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d542f5f-ad89-48ee-99be-bb3dfcba185a admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:21.159276 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd847b4-8f89-40ab-8c15-1b048cbac0ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:21.159924 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd847b4-8f89-40ab-8c15-1b048cbac0ed admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:21.160205 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd847b4-8f89-40ab-8c15-1b048cbac0ed admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:21.160472 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd847b4-8f89-40ab-8c15-1b048cbac0ed admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:21.161103 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd847b4-8f89-40ab-8c15-1b048cbac0ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:21.162707 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd847b4-8f89-40ab-8c15-1b048cbac0ed admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:21.173562 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1188/2388] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:51:21 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:51:21.177787 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf09c6ae-378a-421f-b67d-fd47b3c8b1b6 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:21.189183 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf09c6ae-378a-421f-b67d-fd47b3c8b1b6 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:21.194226 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cb2628-f4ea-48a4-9c15-7b0bfaac481b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:21.195119 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cb2628-f4ea-48a4-9c15-7b0bfaac481b admin admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:21.195406 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cb2628-f4ea-48a4-9c15-7b0bfaac481b admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:21.195712 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cb2628-f4ea-48a4-9c15-7b0bfaac481b admin admin] PATH_INFO: `/v3/projects/47c13d1196e0434cbb5e0b1469625e87/users/55afe027d2b046e094eef686ccbbf5e4/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:21.198261 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cb2628-f4ea-48a4-9c15-7b0bfaac481b admin admin] RBAC: Authorizing `identity:create_grant(project_id=47c13d1196e0434cbb5e0b1469625e87, user_id=55afe027d2b046e094eef686ccbbf5e4, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:21.202121 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cb2628-f4ea-48a4-9c15-7b0bfaac481b admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:21.218799 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1201/2389] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:51:21 2022] PUT /identity/v3/projects/47c13d1196e0434cbb5e0b1469625e87/users/55afe027d2b046e094eef686ccbbf5e4/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:51:21.297839 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd847b4-8f89-40ab-8c15-1b048cbac0ed admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:21.299512 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-7bd847b4-8f89-40ab-8c15-1b048cbac0ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:21.313858 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd847b4-8f89-40ab-8c15-1b048cbac0ed admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:21.384654 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4b88d5-15b7-469d-ba7c-f447e9aa660c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:21.385350 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b88d5-15b7-469d-ba7c-f447e9aa660c admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:21.385760 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b88d5-15b7-469d-ba7c-f447e9aa660c admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:21.385948 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b88d5-15b7-469d-ba7c-f447e9aa660c admin admin] PATH_INFO: `/v3/projects/6ee14e6c9ee5481480f407640b4fbe9f` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:21.387064 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4b88d5-15b7-469d-ba7c-f447e9aa660c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ee14e6c9ee5481480f407640b4fbe9f)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:21.390121 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4b88d5-15b7-469d-ba7c-f447e9aa660c admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:21.447180 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-0a4b88d5-15b7-469d-ba7c-f447e9aa660c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ee14e6c9ee5481480f407640b4fbe9f', 'request_id': 'req-0a4b88d5-15b7-469d-ba7c-f447e9aa660c'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:21.456652 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-0a4b88d5-15b7-469d-ba7c-f447e9aa660c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ee14e6c9ee5481480f407640b4fbe9f', 'request_id': 'req-0a4b88d5-15b7-469d-ba7c-f447e9aa660c'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:21.465738 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-0a4b88d5-15b7-469d-ba7c-f447e9aa660c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ee14e6c9ee5481480f407640b4fbe9f', 'request_id': 'req-0a4b88d5-15b7-469d-ba7c-f447e9aa660c'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:21.478267 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-0a4b88d5-15b7-469d-ba7c-f447e9aa660c admin admin] The token cache is being invalidate because project 6ee14e6c9ee5481480f407640b4fbe9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90653) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:51:21.478585 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-0a4b88d5-15b7-469d-ba7c-f447e9aa660c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:21.479253 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-0a4b88d5-15b7-469d-ba7c-f447e9aa660c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:21.480808 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1189/2390] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:51:21 2022] DELETE /identity/v3/projects/6ee14e6c9ee5481480f407640b4fbe9f => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:51:21.487395 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa6b3cd-61f6-4b98-86e1-973ae10ca846 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:21.487701 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa6b3cd-61f6-4b98-86e1-973ae10ca846 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:21.487990 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa6b3cd-61f6-4b98-86e1-973ae10ca846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:21.631732 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-6aa6b3cd-61f6-4b98-86e1-973ae10ca846 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:51:21.632960 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-6aa6b3cd-61f6-4b98-86e1-973ae10ca846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:21.787287 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1202/2391] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:51:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:51:21.790422 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25008deb-66c0-4ce7-8065-80e42d366504 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:21.790422 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25008deb-66c0-4ce7-8065-80e42d366504 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:21.791050 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25008deb-66c0-4ce7-8065-80e42d366504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:21.811459 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:51:21.821473 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-25008deb-66c0-4ce7-8065-80e42d366504 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:51:21.823117 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-25008deb-66c0-4ce7-8065-80e42d366504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:21.834161 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1190/2392] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:51:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 10:51:21.838752 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa6b3cd-61f6-4b98-86e1-973ae10ca846 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:21.852174 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa6b3cd-61f6-4b98-86e1-973ae10ca846 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:21.878808 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2afb40-d555-4373-9a1b-fe4c6c0f0a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:21.879083 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2afb40-d555-4373-9a1b-fe4c6c0f0a10 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:21.879368 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2afb40-d555-4373-9a1b-fe4c6c0f0a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:21.879666 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2afb40-d555-4373-9a1b-fe4c6c0f0a10 admin admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:21.884446 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2afb40-d555-4373-9a1b-fe4c6c0f0a10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:51:21.884781 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2afb40-d555-4373-9a1b-fe4c6c0f0a10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:21.886432 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2afb40-d555-4373-9a1b-fe4c6c0f0a10 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:21.924629 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1203/2393] 10.222.0.15 () {58 vars in 1153 bytes} [Sun Sep 11 10:51:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:51:21.930894 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25008deb-66c0-4ce7-8065-80e42d366504 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:21.944299 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25008deb-66c0-4ce7-8065-80e42d366504 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:21.949684 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471076bd-aa5c-419a-b2d8-5de13a81af5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:21.950662 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471076bd-aa5c-419a-b2d8-5de13a81af5c admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:21.951323 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471076bd-aa5c-419a-b2d8-5de13a81af5c admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:21.951914 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471076bd-aa5c-419a-b2d8-5de13a81af5c admin admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:21.952745 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471076bd-aa5c-419a-b2d8-5de13a81af5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:21.955524 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471076bd-aa5c-419a-b2d8-5de13a81af5c admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:22.000134 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1191/2394] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:51:21 2022] POST /identity/v3/projects => generated 383 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:51:22.000824 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2afb40-d555-4373-9a1b-fe4c6c0f0a10 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:22.016025 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2afb40-d555-4373-9a1b-fe4c6c0f0a10 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:22.066220 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:51:22.105533 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200f14a1-41e1-4aef-8e05-814b4c30e5a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:22.106306 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200f14a1-41e1-4aef-8e05-814b4c30e5a0 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:22.106766 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200f14a1-41e1-4aef-8e05-814b4c30e5a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:22.108136 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200f14a1-41e1-4aef-8e05-814b4c30e5a0 admin admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:22.109045 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200f14a1-41e1-4aef-8e05-814b4c30e5a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:22.111221 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200f14a1-41e1-4aef-8e05-814b4c30e5a0 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:22.143723 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471076bd-aa5c-419a-b2d8-5de13a81af5c admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:22.154783 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1204/2395] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:51:21 2022] POST /identity/v3/users => generated 292 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:51:22.157340 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471076bd-aa5c-419a-b2d8-5de13a81af5c admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:22.162732 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6d2e22-5fa6-4cef-a3d1-8da7531b14e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:22.163849 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6d2e22-5fa6-4cef-a3d1-8da7531b14e6 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:22.163996 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6d2e22-5fa6-4cef-a3d1-8da7531b14e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:22.164346 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6d2e22-5fa6-4cef-a3d1-8da7531b14e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:22.165230 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6d2e22-5fa6-4cef-a3d1-8da7531b14e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:22.166933 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6d2e22-5fa6-4cef-a3d1-8da7531b14e6 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:22.180645 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1192/2396] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:51:22 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:51:22.184887 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200f14a1-41e1-4aef-8e05-814b4c30e5a0 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:22.199957 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200f14a1-41e1-4aef-8e05-814b4c30e5a0 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:22.206722 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ed8fb0-7b22-47db-a2fb-bdb3117c289b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:22.207824 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed8fb0-7b22-47db-a2fb-bdb3117c289b admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:22.208321 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed8fb0-7b22-47db-a2fb-bdb3117c289b admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:22.208771 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed8fb0-7b22-47db-a2fb-bdb3117c289b admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:22.209733 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ed8fb0-7b22-47db-a2fb-bdb3117c289b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:22.211664 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ed8fb0-7b22-47db-a2fb-bdb3117c289b admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:22.228654 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6d2e22-5fa6-4cef-a3d1-8da7531b14e6 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:22.239048 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1205/2397] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:51:22 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:51:22.242138 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6d2e22-5fa6-4cef-a3d1-8da7531b14e6 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:22.247234 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67d8776-278f-4451-bea8-f55a91148a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:22.248671 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67d8776-278f-4451-bea8-f55a91148a3c admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:22.249166 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67d8776-278f-4451-bea8-f55a91148a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:22.249868 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67d8776-278f-4451-bea8-f55a91148a3c admin admin] PATH_INFO: `/v3/projects/6f100b22066a44c6a581cfa2392cfdae/users/ba5a3b0e83b24d5fa0e1682808fb8f3c/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:22.268113 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67d8776-278f-4451-bea8-f55a91148a3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f100b22066a44c6a581cfa2392cfdae, user_id=ba5a3b0e83b24d5fa0e1682808fb8f3c, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:22.272120 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67d8776-278f-4451-bea8-f55a91148a3c admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:22.323507 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1193/2398] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:51:22 2022] PUT /identity/v3/projects/6f100b22066a44c6a581cfa2392cfdae/users/ba5a3b0e83b24d5fa0e1682808fb8f3c/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:51:22.383673 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c02605-6e0b-49a8-b862-1aa208f500ea None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:22.383938 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c02605-6e0b-49a8-b862-1aa208f500ea None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:22.384223 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c02605-6e0b-49a8-b862-1aa208f500ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:22.415072 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-27c02605-6e0b-49a8-b862-1aa208f500ea None None] MFA Rules not processed for user `ba5a3b0e83b24d5fa0e1682808fb8f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:51:22.416306 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-27c02605-6e0b-49a8-b862-1aa208f500ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:22.492469 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1206/2399] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:51:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:51:22.506099 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67d8776-278f-4451-bea8-f55a91148a3c admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:22.508131 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-b67d8776-278f-4451-bea8-f55a91148a3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:22.581373 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67d8776-278f-4451-bea8-f55a91148a3c admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:22.586938 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada68a69-9e0b-4ae0-a192-a5a30c19ed99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:22.587780 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada68a69-9e0b-4ae0-a192-a5a30c19ed99 service cinder] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:22.588071 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada68a69-9e0b-4ae0-a192-a5a30c19ed99 service cinder] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:22.588426 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada68a69-9e0b-4ae0-a192-a5a30c19ed99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:22.606668 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada68a69-9e0b-4ae0-a192-a5a30c19ed99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:22.609285 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada68a69-9e0b-4ae0-a192-a5a30c19ed99 service cinder] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:22.631455 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1194/2400] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:51:22 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:51:22.649096 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c02605-6e0b-49a8-b862-1aa208f500ea None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:22.754076 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c02605-6e0b-49a8-b862-1aa208f500ea None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:22.767862 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37a3cef-e720-481c-a60c-46ad79516216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:22.768510 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37a3cef-e720-481c-a60c-46ad79516216 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:22.768794 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37a3cef-e720-481c-a60c-46ad79516216 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:22.769062 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37a3cef-e720-481c-a60c-46ad79516216 admin admin] PATH_INFO: `/v3/users/ba5a3b0e83b24d5fa0e1682808fb8f3c` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:22.771151 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37a3cef-e720-481c-a60c-46ad79516216 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba5a3b0e83b24d5fa0e1682808fb8f3c)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:22.773268 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37a3cef-e720-481c-a60c-46ad79516216 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:22.856549 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:51:22.860304 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-d37a3cef-e720-481c-a60c-46ad79516216 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba5a3b0e83b24d5fa0e1682808fb8f3c'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:22.873022 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-d37a3cef-e720-481c-a60c-46ad79516216 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba5a3b0e83b24d5fa0e1682808fb8f3c'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:22.874287 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-d37a3cef-e720-481c-a60c-46ad79516216 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba5a3b0e83b24d5fa0e1682808fb8f3c'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:22.875617 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-d37a3cef-e720-481c-a60c-46ad79516216 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba5a3b0e83b24d5fa0e1682808fb8f3c'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:22.904019 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-d37a3cef-e720-481c-a60c-46ad79516216 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba5a3b0e83b24d5fa0e1682808fb8f3c'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:22.921797 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1207/2401] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:51:22 2022] DELETE /identity/v3/users/ba5a3b0e83b24d5fa0e1682808fb8f3c => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:51:22.927470 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da43aa51-fdf6-4f6b-b524-4156062b0ce8 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:22.927964 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da43aa51-fdf6-4f6b-b524-4156062b0ce8 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:22.928083 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da43aa51-fdf6-4f6b-b524-4156062b0ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:22.960656 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-da43aa51-fdf6-4f6b-b524-4156062b0ce8 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:51:22.962059 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-da43aa51-fdf6-4f6b-b524-4156062b0ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:23.083195 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1195/2402] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:51:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:51:23.100501 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37a3cef-e720-481c-a60c-46ad79516216 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:23.101908 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-d37a3cef-e720-481c-a60c-46ad79516216 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:23.239365 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37a3cef-e720-481c-a60c-46ad79516216 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:23.246716 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2545fc08-0aae-4fae-8184-edd75ef8a590 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:23.246871 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2545fc08-0aae-4fae-8184-edd75ef8a590 service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:23.247101 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2545fc08-0aae-4fae-8184-edd75ef8a590 service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:23.248105 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2545fc08-0aae-4fae-8184-edd75ef8a590 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:23.265448 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2545fc08-0aae-4fae-8184-edd75ef8a590 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:23.266903 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2545fc08-0aae-4fae-8184-edd75ef8a590 service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:23.286214 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1208/2403] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:51:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:51:23.897616 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da43aa51-fdf6-4f6b-b524-4156062b0ce8 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:23.901410 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-da43aa51-fdf6-4f6b-b524-4156062b0ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:23.946485 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da43aa51-fdf6-4f6b-b524-4156062b0ce8 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:23.957632 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14b48e7-d086-40f1-be9e-6cc55fd1265a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:23.958300 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b48e7-d086-40f1-be9e-6cc55fd1265a admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:23.958585 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b48e7-d086-40f1-be9e-6cc55fd1265a admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:23.958862 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b48e7-d086-40f1-be9e-6cc55fd1265a admin admin] PATH_INFO: `/v3/projects/6f100b22066a44c6a581cfa2392cfdae` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:23.965721 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14b48e7-d086-40f1-be9e-6cc55fd1265a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f100b22066a44c6a581cfa2392cfdae)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:23.968473 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14b48e7-d086-40f1-be9e-6cc55fd1265a admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:24.058401 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:51:24.076841 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-a14b48e7-d086-40f1-be9e-6cc55fd1265a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f100b22066a44c6a581cfa2392cfdae', 'request_id': 'req-a14b48e7-d086-40f1-be9e-6cc55fd1265a'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:24.086008 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-a14b48e7-d086-40f1-be9e-6cc55fd1265a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f100b22066a44c6a581cfa2392cfdae', 'request_id': 'req-a14b48e7-d086-40f1-be9e-6cc55fd1265a'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:24.100312 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-a14b48e7-d086-40f1-be9e-6cc55fd1265a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f100b22066a44c6a581cfa2392cfdae', 'request_id': 'req-a14b48e7-d086-40f1-be9e-6cc55fd1265a'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:24.120282 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-a14b48e7-d086-40f1-be9e-6cc55fd1265a admin admin] The token cache is being invalidate because project 6f100b22066a44c6a581cfa2392cfdae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90653) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:51:24.120647 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-a14b48e7-d086-40f1-be9e-6cc55fd1265a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:24.121526 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-a14b48e7-d086-40f1-be9e-6cc55fd1265a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:24.127209 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1196/2404] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:51:23 2022] DELETE /identity/v3/projects/6f100b22066a44c6a581cfa2392cfdae => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:51:24.128892 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f92d48-aee1-47c7-be40-30e470d41efb None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:24.129339 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f92d48-aee1-47c7-be40-30e470d41efb None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:24.129444 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f92d48-aee1-47c7-be40-30e470d41efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:24.167387 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-c0f92d48-aee1-47c7-be40-30e470d41efb None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:51:24.168667 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-c0f92d48-aee1-47c7-be40-30e470d41efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:24.263215 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1209/2405] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:51:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:51:24.270670 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983a2f6e-f263-4a45-9639-c651ec615bfe None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:24.271199 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983a2f6e-f263-4a45-9639-c651ec615bfe None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:24.271672 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983a2f6e-f263-4a45-9639-c651ec615bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:24.328958 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-983a2f6e-f263-4a45-9639-c651ec615bfe None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:51:24.330374 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-983a2f6e-f263-4a45-9639-c651ec615bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:24.345477 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1197/2406] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:51:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:51:24.346816 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f92d48-aee1-47c7-be40-30e470d41efb None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:24.363317 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f92d48-aee1-47c7-be40-30e470d41efb None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:24.368891 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052c18b6-f246-4fc4-a29c-dea9c4aa822e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:24.369464 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052c18b6-f246-4fc4-a29c-dea9c4aa822e admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:24.369747 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052c18b6-f246-4fc4-a29c-dea9c4aa822e admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:24.370022 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052c18b6-f246-4fc4-a29c-dea9c4aa822e admin admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:24.370784 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052c18b6-f246-4fc4-a29c-dea9c4aa822e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:51:24.371868 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052c18b6-f246-4fc4-a29c-dea9c4aa822e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:24.374078 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052c18b6-f246-4fc4-a29c-dea9c4aa822e admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:24.396792 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1210/2407] 10.222.0.15 () {58 vars in 1153 bytes} [Sun Sep 11 10:51:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:51:24.398723 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983a2f6e-f263-4a45-9639-c651ec615bfe None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:24.411092 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983a2f6e-f263-4a45-9639-c651ec615bfe None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:24.432660 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f3b4da-18ed-4ccf-8f8a-3735779c5254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:24.433600 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f3b4da-18ed-4ccf-8f8a-3735779c5254 admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:24.434063 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f3b4da-18ed-4ccf-8f8a-3735779c5254 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:24.434510 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f3b4da-18ed-4ccf-8f8a-3735779c5254 admin admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:24.438818 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f3b4da-18ed-4ccf-8f8a-3735779c5254 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:24.442891 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f3b4da-18ed-4ccf-8f8a-3735779c5254 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:24.483221 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1198/2408] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:51:24 2022] POST /identity/v3/projects => generated 383 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:51:24.485108 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052c18b6-f246-4fc4-a29c-dea9c4aa822e admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:24.496497 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052c18b6-f246-4fc4-a29c-dea9c4aa822e admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:24.578544 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:51:24.600891 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca438f6-1689-42f1-919e-a3bc196176f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:24.601317 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca438f6-1689-42f1-919e-a3bc196176f1 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:24.601696 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca438f6-1689-42f1-919e-a3bc196176f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:24.602206 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca438f6-1689-42f1-919e-a3bc196176f1 admin admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:24.602853 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca438f6-1689-42f1-919e-a3bc196176f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:24.605376 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca438f6-1689-42f1-919e-a3bc196176f1 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:24.642221 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1211/2409] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:51:24 2022] POST /identity/v3/users => generated 292 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:51:24.647580 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f3b4da-18ed-4ccf-8f8a-3735779c5254 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:24.660890 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f3b4da-18ed-4ccf-8f8a-3735779c5254 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:24.666257 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22397cc6-5712-435f-ad99-94d833dc39cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:24.667450 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22397cc6-5712-435f-ad99-94d833dc39cb admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:24.667783 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22397cc6-5712-435f-ad99-94d833dc39cb admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:24.668056 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22397cc6-5712-435f-ad99-94d833dc39cb admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:24.668764 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22397cc6-5712-435f-ad99-94d833dc39cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:24.671763 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22397cc6-5712-435f-ad99-94d833dc39cb admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:24.690889 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1199/2410] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:51:24 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:51:24.692185 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca438f6-1689-42f1-919e-a3bc196176f1 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:24.703407 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca438f6-1689-42f1-919e-a3bc196176f1 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:24.709397 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1efac7a-1d1f-44eb-ba64-84d2dca55ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:24.710459 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1efac7a-1d1f-44eb-ba64-84d2dca55ec8 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:24.717505 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1efac7a-1d1f-44eb-ba64-84d2dca55ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:24.718191 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1efac7a-1d1f-44eb-ba64-84d2dca55ec8 admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:24.719155 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1efac7a-1d1f-44eb-ba64-84d2dca55ec8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:24.721249 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1efac7a-1d1f-44eb-ba64-84d2dca55ec8 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:24.737623 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1212/2411] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:51:24 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:51:24.741713 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22397cc6-5712-435f-ad99-94d833dc39cb admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:24.758637 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22397cc6-5712-435f-ad99-94d833dc39cb admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:24.768606 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b468790-959b-49b9-91ed-7915835508f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:24.769343 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b468790-959b-49b9-91ed-7915835508f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:24.769623 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b468790-959b-49b9-91ed-7915835508f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:24.769893 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b468790-959b-49b9-91ed-7915835508f9 admin admin] PATH_INFO: `/v3/projects/297fbd764a13458dab22ff654dc8ff00/users/de42a3740a9d440c9a9235c2d7e1873d/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:24.792104 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b468790-959b-49b9-91ed-7915835508f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=297fbd764a13458dab22ff654dc8ff00, user_id=de42a3740a9d440c9a9235c2d7e1873d, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:24.796063 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b468790-959b-49b9-91ed-7915835508f9 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:24.817285 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1200/2412] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:51:24 2022] PUT /identity/v3/projects/297fbd764a13458dab22ff654dc8ff00/users/de42a3740a9d440c9a9235c2d7e1873d/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:51:24.832916 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa12311-fa93-4279-aeb3-d5ba7847f3b3 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:24.833268 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa12311-fa93-4279-aeb3-d5ba7847f3b3 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:24.833548 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa12311-fa93-4279-aeb3-d5ba7847f3b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:24.901570 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-1fa12311-fa93-4279-aeb3-d5ba7847f3b3 None None] MFA Rules not processed for user `de42a3740a9d440c9a9235c2d7e1873d`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:51:24.903012 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-1fa12311-fa93-4279-aeb3-d5ba7847f3b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:25.039100 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1213/2413] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:51:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:51:25.053345 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b468790-959b-49b9-91ed-7915835508f9 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:25.055873 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-9b468790-959b-49b9-91ed-7915835508f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:25.146968 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b468790-959b-49b9-91ed-7915835508f9 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:25.151205 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0ed8e6-a294-4452-a9f9-bd83ddbfa4b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:25.151869 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ed8e6-a294-4452-a9f9-bd83ddbfa4b6 service cinder] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:25.152143 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ed8e6-a294-4452-a9f9-bd83ddbfa4b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:25.152419 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ed8e6-a294-4452-a9f9-bd83ddbfa4b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:25.167280 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0ed8e6-a294-4452-a9f9-bd83ddbfa4b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:25.168787 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0ed8e6-a294-4452-a9f9-bd83ddbfa4b6 service cinder] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:25.186966 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1201/2414] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:51:25 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:51:26.693294 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9da2fb8-8880-4a98-8dcc-bb73fa94e9b2 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:26.694113 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9da2fb8-8880-4a98-8dcc-bb73fa94e9b2 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:26.695053 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9da2fb8-8880-4a98-8dcc-bb73fa94e9b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:26.736771 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-d9da2fb8-8880-4a98-8dcc-bb73fa94e9b2 None None] MFA Rules not processed for user `55afe027d2b046e094eef686ccbbf5e4`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:51:26.738208 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-d9da2fb8-8880-4a98-8dcc-bb73fa94e9b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:26.829621 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1214/2415] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:51:26 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:51:26.842585 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0ed8e6-a294-4452-a9f9-bd83ddbfa4b6 service cinder] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:26.844214 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-cf0ed8e6-a294-4452-a9f9-bd83ddbfa4b6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:26.915902 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0ed8e6-a294-4452-a9f9-bd83ddbfa4b6 service cinder] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:26.920098 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cade543b-b582-4171-9153-3be97483ae31 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1632da5fbe384b5c849be1d4d86c0f1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:26.920936 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade543b-b582-4171-9153-3be97483ae31 service glance] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:26.921342 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade543b-b582-4171-9153-3be97483ae31 service glance] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:26.921792 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade543b-b582-4171-9153-3be97483ae31 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:26.923628 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-cade543b-b582-4171-9153-3be97483ae31 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:26.941895 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:51:26.987092 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cade543b-b582-4171-9153-3be97483ae31 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:26.988575 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cade543b-b582-4171-9153-3be97483ae31 service glance] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:26.997885 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1202/2416] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:51:26 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:51:27.165718 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5828e15-f411-43a3-a064-98590e36855c None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:27.166010 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5828e15-f411-43a3-a064-98590e36855c None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:27.166307 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5828e15-f411-43a3-a064-98590e36855c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:27.188486 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-b5828e15-f411-43a3-a064-98590e36855c None None] MFA Rules not processed for user `bb44e7aef0c7436d93a7233307b5f095`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:51:27.190408 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-b5828e15-f411-43a3-a064-98590e36855c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:27.243616 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:51:27.289414 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1215/2417] 10.222.0.15 () {58 vars in 1009 bytes} [Sun Sep 11 10:51:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:51:27.302983 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cade543b-b582-4171-9153-3be97483ae31 service glance] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:27.304478 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-cade543b-b582-4171-9153-3be97483ae31 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:27.389666 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cade543b-b582-4171-9153-3be97483ae31 service glance] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:27.397593 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069f0fa9-56c6-40c2-a55e-31d18bac52c2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7ca566e66a45b29e3d02f47bf84981', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:27.398483 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f0fa9-56c6-40c2-a55e-31d18bac52c2 service swift] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:27.398959 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f0fa9-56c6-40c2-a55e-31d18bac52c2 service swift] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:27.399454 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f0fa9-56c6-40c2-a55e-31d18bac52c2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:27.408571 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069f0fa9-56c6-40c2-a55e-31d18bac52c2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:27.410533 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069f0fa9-56c6-40c2-a55e-31d18bac52c2 service swift] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:27.419692 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1203/2418] 10.222.0.15 () {60 vars in 1384 bytes} [Sun Sep 11 10:51:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 10:51:27.936692 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5828e15-f411-43a3-a064-98590e36855c None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:28.022725 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5828e15-f411-43a3-a064-98590e36855c None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:28.031262 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c4e55d-b89f-4f26-9f54-e113189f2c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:28.032364 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c4e55d-b89f-4f26-9f54-e113189f2c8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:28.032701 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c4e55d-b89f-4f26-9f54-e113189f2c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:28.033041 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c4e55d-b89f-4f26-9f54-e113189f2c8c admin admin] PATH_INFO: `/v3/users/de42a3740a9d440c9a9235c2d7e1873d` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:28.034258 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c4e55d-b89f-4f26-9f54-e113189f2c8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=de42a3740a9d440c9a9235c2d7e1873d)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:28.036779 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c4e55d-b89f-4f26-9f54-e113189f2c8c admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:28.107372 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-04c4e55d-b89f-4f26-9f54-e113189f2c8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de42a3740a9d440c9a9235c2d7e1873d'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:28.121344 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-04c4e55d-b89f-4f26-9f54-e113189f2c8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de42a3740a9d440c9a9235c2d7e1873d'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:28.122794 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-04c4e55d-b89f-4f26-9f54-e113189f2c8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de42a3740a9d440c9a9235c2d7e1873d'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:28.123636 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-04c4e55d-b89f-4f26-9f54-e113189f2c8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de42a3740a9d440c9a9235c2d7e1873d'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:28.149234 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-04c4e55d-b89f-4f26-9f54-e113189f2c8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de42a3740a9d440c9a9235c2d7e1873d'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:28.172818 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1216/2419] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:51:27 2022] DELETE /identity/v3/users/de42a3740a9d440c9a9235c2d7e1873d => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:51:28.177180 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09b5bbb-cb78-47bf-a2da-e2b7b128e5b2 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:28.178078 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09b5bbb-cb78-47bf-a2da-e2b7b128e5b2 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:28.178624 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09b5bbb-cb78-47bf-a2da-e2b7b128e5b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:28.212264 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-c09b5bbb-cb78-47bf-a2da-e2b7b128e5b2 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:51:28.213398 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-c09b5bbb-cb78-47bf-a2da-e2b7b128e5b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:28.309416 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1204/2420] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:51:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:51:28.321391 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c4e55d-b89f-4f26-9f54-e113189f2c8c admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:28.322910 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-04c4e55d-b89f-4f26-9f54-e113189f2c8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:28.401909 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c4e55d-b89f-4f26-9f54-e113189f2c8c admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:28.406839 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b856bbc-b9d1-4dc8-8f25-c93a55a9898e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:28.407960 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b856bbc-b9d1-4dc8-8f25-c93a55a9898e service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:28.408479 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b856bbc-b9d1-4dc8-8f25-c93a55a9898e service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:28.409274 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b856bbc-b9d1-4dc8-8f25-c93a55a9898e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:28.437361 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b856bbc-b9d1-4dc8-8f25-c93a55a9898e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:28.441116 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b856bbc-b9d1-4dc8-8f25-c93a55a9898e service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:28.454315 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1217/2421] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:51:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:51:28.811114 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09b5bbb-cb78-47bf-a2da-e2b7b128e5b2 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:28.813112 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-c09b5bbb-cb78-47bf-a2da-e2b7b128e5b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:28.836905 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09b5bbb-cb78-47bf-a2da-e2b7b128e5b2 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:28.848192 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdf78fb-fa07-4843-9ea0-413240bc5dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:28.849193 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf78fb-fa07-4843-9ea0-413240bc5dc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:28.850082 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf78fb-fa07-4843-9ea0-413240bc5dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:28.850575 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf78fb-fa07-4843-9ea0-413240bc5dc1 admin admin] PATH_INFO: `/v3/projects/297fbd764a13458dab22ff654dc8ff00` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:28.851908 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdf78fb-fa07-4843-9ea0-413240bc5dc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=297fbd764a13458dab22ff654dc8ff00)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:28.854198 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdf78fb-fa07-4843-9ea0-413240bc5dc1 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:28.922980 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:51:28.928685 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-fbdf78fb-fa07-4843-9ea0-413240bc5dc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '297fbd764a13458dab22ff654dc8ff00', 'request_id': 'req-fbdf78fb-fa07-4843-9ea0-413240bc5dc1'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:28.940290 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-fbdf78fb-fa07-4843-9ea0-413240bc5dc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '297fbd764a13458dab22ff654dc8ff00', 'request_id': 'req-fbdf78fb-fa07-4843-9ea0-413240bc5dc1'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:28.960332 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-fbdf78fb-fa07-4843-9ea0-413240bc5dc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '297fbd764a13458dab22ff654dc8ff00', 'request_id': 'req-fbdf78fb-fa07-4843-9ea0-413240bc5dc1'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:28.974178 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-fbdf78fb-fa07-4843-9ea0-413240bc5dc1 admin admin] The token cache is being invalidate because project 297fbd764a13458dab22ff654dc8ff00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90653) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:51:28.974481 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-fbdf78fb-fa07-4843-9ea0-413240bc5dc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:28.975324 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-fbdf78fb-fa07-4843-9ea0-413240bc5dc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:28.976929 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1205/2422] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:51:28 2022] DELETE /identity/v3/projects/297fbd764a13458dab22ff654dc8ff00 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:51:28.983126 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f4cd59-6cf8-402a-8e79-f9f72b72abef None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:28.983126 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f4cd59-6cf8-402a-8e79-f9f72b72abef None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:28.983720 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f4cd59-6cf8-402a-8e79-f9f72b72abef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:29.006407 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-07f4cd59-6cf8-402a-8e79-f9f72b72abef None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:51:29.006407 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-07f4cd59-6cf8-402a-8e79-f9f72b72abef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:29.017375 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:51:29.079469 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1218/2423] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:51:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:51:29.088504 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c35af49-cfe0-4a10-bd1d-b159df6dd375 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:29.088797 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c35af49-cfe0-4a10-bd1d-b159df6dd375 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:29.089079 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c35af49-cfe0-4a10-bd1d-b159df6dd375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:29.114096 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-3c35af49-cfe0-4a10-bd1d-b159df6dd375 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:51:29.115668 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-3c35af49-cfe0-4a10-bd1d-b159df6dd375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:29.128987 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1206/2424] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:51:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:51:29.129879 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f4cd59-6cf8-402a-8e79-f9f72b72abef None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:29.141993 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f4cd59-6cf8-402a-8e79-f9f72b72abef None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:29.148299 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74885e80-f73c-4a4f-b946-422e7fe60dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:29.149139 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74885e80-f73c-4a4f-b946-422e7fe60dc6 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:29.150062 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74885e80-f73c-4a4f-b946-422e7fe60dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:29.150340 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74885e80-f73c-4a4f-b946-422e7fe60dc6 admin admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:29.150986 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74885e80-f73c-4a4f-b946-422e7fe60dc6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:51:29.151286 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74885e80-f73c-4a4f-b946-422e7fe60dc6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:29.152935 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74885e80-f73c-4a4f-b946-422e7fe60dc6 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:29.171605 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1219/2425] 10.222.0.15 () {58 vars in 1153 bytes} [Sun Sep 11 10:51:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:51:29.174946 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c35af49-cfe0-4a10-bd1d-b159df6dd375 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:29.187064 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c35af49-cfe0-4a10-bd1d-b159df6dd375 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:29.193269 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93c0fb4-0513-4415-8bcd-5a3d8fd68210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:29.194011 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93c0fb4-0513-4415-8bcd-5a3d8fd68210 admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:29.194394 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93c0fb4-0513-4415-8bcd-5a3d8fd68210 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:29.194816 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93c0fb4-0513-4415-8bcd-5a3d8fd68210 admin admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:29.195725 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93c0fb4-0513-4415-8bcd-5a3d8fd68210 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:29.197863 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93c0fb4-0513-4415-8bcd-5a3d8fd68210 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:29.232598 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1207/2426] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:51:29 2022] POST /identity/v3/projects => generated 375 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 10:51:29.238382 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74885e80-f73c-4a4f-b946-422e7fe60dc6 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:29.249963 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74885e80-f73c-4a4f-b946-422e7fe60dc6 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:29.328141 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c054e69f-0774-445c-8bc0-c87b10a899c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:29.328761 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c054e69f-0774-445c-8bc0-c87b10a899c0 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:29.329072 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c054e69f-0774-445c-8bc0-c87b10a899c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:29.329314 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c054e69f-0774-445c-8bc0-c87b10a899c0 admin admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:29.329946 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c054e69f-0774-445c-8bc0-c87b10a899c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:29.331893 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c054e69f-0774-445c-8bc0-c87b10a899c0 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:29.358115 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1220/2427] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:51:29 2022] POST /identity/v3/users => generated 288 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:51:29.362228 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93c0fb4-0513-4415-8bcd-5a3d8fd68210 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:29.373948 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93c0fb4-0513-4415-8bcd-5a3d8fd68210 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:29.380798 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b67ffc-340b-4a18-ad4d-560340bbb95a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:29.381711 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b67ffc-340b-4a18-ad4d-560340bbb95a admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:29.382348 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b67ffc-340b-4a18-ad4d-560340bbb95a admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:29.382689 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b67ffc-340b-4a18-ad4d-560340bbb95a admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:29.383408 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b67ffc-340b-4a18-ad4d-560340bbb95a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:29.385205 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b67ffc-340b-4a18-ad4d-560340bbb95a admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:29.399479 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1208/2428] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:51:29 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 10:51:29.400875 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c054e69f-0774-445c-8bc0-c87b10a899c0 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:29.412312 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c054e69f-0774-445c-8bc0-c87b10a899c0 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:29.417194 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea12781-e6d8-4130-9d00-5a2ded4cfb6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:29.417808 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea12781-e6d8-4130-9d00-5a2ded4cfb6f admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:29.422289 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea12781-e6d8-4130-9d00-5a2ded4cfb6f admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:29.422629 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea12781-e6d8-4130-9d00-5a2ded4cfb6f admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:29.423311 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea12781-e6d8-4130-9d00-5a2ded4cfb6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:29.425074 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea12781-e6d8-4130-9d00-5a2ded4cfb6f admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:29.437308 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1221/2429] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:51:29 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:51:29.443477 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b67ffc-340b-4a18-ad4d-560340bbb95a admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:29.454104 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b67ffc-340b-4a18-ad4d-560340bbb95a admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:29.459347 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bee882f-5b02-4f89-82c9-7e547b451c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:29.462423 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee882f-5b02-4f89-82c9-7e547b451c1b admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:29.462423 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee882f-5b02-4f89-82c9-7e547b451c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:29.462423 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee882f-5b02-4f89-82c9-7e547b451c1b admin admin] PATH_INFO: `/v3/projects/c3bd7d12d67b4009a16dcf11d929ca50/users/246bee549b6f4b98aa3d848ce5eb2622/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:29.478209 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bee882f-5b02-4f89-82c9-7e547b451c1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3bd7d12d67b4009a16dcf11d929ca50, user_id=246bee549b6f4b98aa3d848ce5eb2622, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:29.483460 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bee882f-5b02-4f89-82c9-7e547b451c1b admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:29.496718 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1209/2430] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:51:29 2022] PUT /identity/v3/projects/c3bd7d12d67b4009a16dcf11d929ca50/users/246bee549b6f4b98aa3d848ce5eb2622/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:51:29.508136 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed1ecb9-19b9-4c62-b0b3-17d7fb65ba06 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:29.508456 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed1ecb9-19b9-4c62-b0b3-17d7fb65ba06 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:29.508699 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed1ecb9-19b9-4c62-b0b3-17d7fb65ba06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:29.541469 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-5ed1ecb9-19b9-4c62-b0b3-17d7fb65ba06 None None] MFA Rules not processed for user `246bee549b6f4b98aa3d848ce5eb2622`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:51:29.546275 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-5ed1ecb9-19b9-4c62-b0b3-17d7fb65ba06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:29.621150 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1222/2431] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:51:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:51:29.633448 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bee882f-5b02-4f89-82c9-7e547b451c1b admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:29.635207 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-6bee882f-5b02-4f89-82c9-7e547b451c1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:29.740794 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bee882f-5b02-4f89-82c9-7e547b451c1b admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:29.758448 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baef0571-0943-494b-b926-47afe805de31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:29.758448 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef0571-0943-494b-b926-47afe805de31 service cinder] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:29.758448 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef0571-0943-494b-b926-47afe805de31 service cinder] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:29.758448 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef0571-0943-494b-b926-47afe805de31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:29.758948 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baef0571-0943-494b-b926-47afe805de31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:29.760593 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baef0571-0943-494b-b926-47afe805de31 service cinder] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:29.773363 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1210/2432] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:51:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:51:30.342401 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed1ecb9-19b9-4c62-b0b3-17d7fb65ba06 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:30.349283 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-5ed1ecb9-19b9-4c62-b0b3-17d7fb65ba06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:30.363057 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:51:30.438202 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed1ecb9-19b9-4c62-b0b3-17d7fb65ba06 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:30.443292 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66c26a2-4459-4146-964d-c69e25efe425 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:30.444312 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66c26a2-4459-4146-964d-c69e25efe425 service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:30.444776 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66c26a2-4459-4146-964d-c69e25efe425 service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:30.445116 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66c26a2-4459-4146-964d-c69e25efe425 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:30.447265 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-d66c26a2-4459-4146-964d-c69e25efe425 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:30.572080 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66c26a2-4459-4146-964d-c69e25efe425 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:30.573663 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66c26a2-4459-4146-964d-c69e25efe425 service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:30.587038 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1223/2433] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:51:30 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:51:38.433694 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e527d6-9e1a-48fc-95e7-fa278b244e52 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:38.434899 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e527d6-9e1a-48fc-95e7-fa278b244e52 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:38.434899 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e527d6-9e1a-48fc-95e7-fa278b244e52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:38.454243 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-d6e527d6-9e1a-48fc-95e7-fa278b244e52 None None] MFA Rules not processed for user `8902fe9b4ce54ad398422722101a947a`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:51:38.455483 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-d6e527d6-9e1a-48fc-95e7-fa278b244e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:38.512988 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1211/2434] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:51:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:51:38.549533 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66c26a2-4459-4146-964d-c69e25efe425 service neutron] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:38.551063 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-d66c26a2-4459-4146-964d-c69e25efe425 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:38.580569 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e527d6-9e1a-48fc-95e7-fa278b244e52 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:38.590612 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e527d6-9e1a-48fc-95e7-fa278b244e52 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:38.601083 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c59fef1-2a0a-4e10-b3e7-fcc75a16404d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:38.601708 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c59fef1-2a0a-4e10-b3e7-fcc75a16404d service cinder] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:38.601986 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c59fef1-2a0a-4e10-b3e7-fcc75a16404d service cinder] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:38.602255 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c59fef1-2a0a-4e10-b3e7-fcc75a16404d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:38.603992 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-7c59fef1-2a0a-4e10-b3e7-fcc75a16404d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:38.609181 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66c26a2-4459-4146-964d-c69e25efe425 service neutron] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:38.612775 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eaa780-d63f-4c4a-8d4d-a154932e1d17 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1632da5fbe384b5c849be1d4d86c0f1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:38.613378 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eaa780-d63f-4c4a-8d4d-a154932e1d17 service glance] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:38.613656 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eaa780-d63f-4c4a-8d4d-a154932e1d17 service glance] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:38.613927 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eaa780-d63f-4c4a-8d4d-a154932e1d17 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:38.615673 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-52eaa780-d63f-4c4a-8d4d-a154932e1d17 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:38.619048 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:51:38.682021 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c59fef1-2a0a-4e10-b3e7-fcc75a16404d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:38.683486 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c59fef1-2a0a-4e10-b3e7-fcc75a16404d service cinder] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:38.688135 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eaa780-d63f-4c4a-8d4d-a154932e1d17 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:38.689719 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eaa780-d63f-4c4a-8d4d-a154932e1d17 service glance] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:38.694773 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1212/2435] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:51:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 10:51:38.695648 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=100 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:51:38.702371 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1224/2436] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:51:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:51:41.291703 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc01548-4e22-40d9-bf63-6574b69a70d8 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:41.292243 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc01548-4e22-40d9-bf63-6574b69a70d8 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:41.292348 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc01548-4e22-40d9-bf63-6574b69a70d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:41.317698 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-1cc01548-4e22-40d9-bf63-6574b69a70d8 None None] MFA Rules not processed for user `bb44e7aef0c7436d93a7233307b5f095`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:51:41.319009 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-1cc01548-4e22-40d9-bf63-6574b69a70d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:41.393467 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1213/2437] 10.222.0.15 () {58 vars in 1009 bytes} [Sun Sep 11 10:51:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:51:41.407366 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eaa780-d63f-4c4a-8d4d-a154932e1d17 service glance] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:41.408836 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-52eaa780-d63f-4c4a-8d4d-a154932e1d17 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:41.482480 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eaa780-d63f-4c4a-8d4d-a154932e1d17 service glance] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:41.488982 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc114c7a-c258-41c2-8893-a896eb4695d9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7ca566e66a45b29e3d02f47bf84981', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:41.489597 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc114c7a-c258-41c2-8893-a896eb4695d9 service swift] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:41.489880 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc114c7a-c258-41c2-8893-a896eb4695d9 service swift] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:41.490150 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc114c7a-c258-41c2-8893-a896eb4695d9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:41.501032 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc114c7a-c258-41c2-8893-a896eb4695d9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:41.503089 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc114c7a-c258-41c2-8893-a896eb4695d9 service swift] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:41.515609 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1225/2438] 10.222.0.15 () {60 vars in 1384 bytes} [Sun Sep 11 10:51:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 10:51:42.415936 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc01548-4e22-40d9-bf63-6574b69a70d8 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:42.479298 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc01548-4e22-40d9-bf63-6574b69a70d8 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:42.484676 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e4cb7c-42aa-4caa-ba55-1ec43bd61416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:42.485297 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e4cb7c-42aa-4caa-ba55-1ec43bd61416 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:42.485574 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e4cb7c-42aa-4caa-ba55-1ec43bd61416 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:42.485846 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e4cb7c-42aa-4caa-ba55-1ec43bd61416 admin admin] PATH_INFO: `/v3/users/246bee549b6f4b98aa3d848ce5eb2622` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:42.486950 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e4cb7c-42aa-4caa-ba55-1ec43bd61416 admin admin] RBAC: Authorizing `identity:delete_user(user_id=246bee549b6f4b98aa3d848ce5eb2622)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:42.488893 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e4cb7c-42aa-4caa-ba55-1ec43bd61416 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:42.555737 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-07e4cb7c-42aa-4caa-ba55-1ec43bd61416 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '246bee549b6f4b98aa3d848ce5eb2622'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:42.558414 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:51:42.569928 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-07e4cb7c-42aa-4caa-ba55-1ec43bd61416 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '246bee549b6f4b98aa3d848ce5eb2622'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:42.570794 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-07e4cb7c-42aa-4caa-ba55-1ec43bd61416 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '246bee549b6f4b98aa3d848ce5eb2622'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:42.571844 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-07e4cb7c-42aa-4caa-ba55-1ec43bd61416 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '246bee549b6f4b98aa3d848ce5eb2622'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:42.580512 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-07e4cb7c-42aa-4caa-ba55-1ec43bd61416 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '246bee549b6f4b98aa3d848ce5eb2622'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:42.599340 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1214/2439] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:51:42 2022] DELETE /identity/v3/users/246bee549b6f4b98aa3d848ce5eb2622 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:51:42.604016 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac9521e-9eac-4a02-8df4-68c519bbc421 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:42.604480 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac9521e-9eac-4a02-8df4-68c519bbc421 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:42.604582 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac9521e-9eac-4a02-8df4-68c519bbc421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:42.623947 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-3ac9521e-9eac-4a02-8df4-68c519bbc421 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:51:42.625139 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-3ac9521e-9eac-4a02-8df4-68c519bbc421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:42.699384 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1226/2440] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:51:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:51:42.709168 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e4cb7c-42aa-4caa-ba55-1ec43bd61416 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:42.710674 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-07e4cb7c-42aa-4caa-ba55-1ec43bd61416 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:42.786183 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e4cb7c-42aa-4caa-ba55-1ec43bd61416 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:42.789907 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb565afe-9120-4e1e-900c-fa3de271bdbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:42.790708 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb565afe-9120-4e1e-900c-fa3de271bdbb service neutron] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:42.791139 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb565afe-9120-4e1e-900c-fa3de271bdbb service neutron] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:42.791440 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb565afe-9120-4e1e-900c-fa3de271bdbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:42.804873 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb565afe-9120-4e1e-900c-fa3de271bdbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:42.806301 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb565afe-9120-4e1e-900c-fa3de271bdbb service neutron] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:42.816521 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1215/2441] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:51:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:51:43.111805 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac9521e-9eac-4a02-8df4-68c519bbc421 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:43.113346 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-3ac9521e-9eac-4a02-8df4-68c519bbc421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:43.128949 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac9521e-9eac-4a02-8df4-68c519bbc421 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:43.134011 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c38e07-fe4b-4de8-989c-b275f5162277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:43.134655 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c38e07-fe4b-4de8-989c-b275f5162277 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:43.135491 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c38e07-fe4b-4de8-989c-b275f5162277 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:43.135783 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c38e07-fe4b-4de8-989c-b275f5162277 admin admin] PATH_INFO: `/v3/projects/c3bd7d12d67b4009a16dcf11d929ca50` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:43.137328 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c38e07-fe4b-4de8-989c-b275f5162277 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3bd7d12d67b4009a16dcf11d929ca50)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:43.139237 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c38e07-fe4b-4de8-989c-b275f5162277 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:43.186726 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-80c38e07-fe4b-4de8-989c-b275f5162277 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3bd7d12d67b4009a16dcf11d929ca50', 'request_id': 'req-80c38e07-fe4b-4de8-989c-b275f5162277'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:43.193107 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-80c38e07-fe4b-4de8-989c-b275f5162277 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3bd7d12d67b4009a16dcf11d929ca50', 'request_id': 'req-80c38e07-fe4b-4de8-989c-b275f5162277'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:43.198822 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-80c38e07-fe4b-4de8-989c-b275f5162277 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3bd7d12d67b4009a16dcf11d929ca50', 'request_id': 'req-80c38e07-fe4b-4de8-989c-b275f5162277'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:43.211658 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-80c38e07-fe4b-4de8-989c-b275f5162277 admin admin] The token cache is being invalidate because project c3bd7d12d67b4009a16dcf11d929ca50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:51:43.211954 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-80c38e07-fe4b-4de8-989c-b275f5162277 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:43.212579 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-80c38e07-fe4b-4de8-989c-b275f5162277 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:51:43.214061 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1227/2442] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:51:43 2022] DELETE /identity/v3/projects/c3bd7d12d67b4009a16dcf11d929ca50 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:51:44.315463 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb565afe-9120-4e1e-900c-fa3de271bdbb service neutron] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:44.316916 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-cb565afe-9120-4e1e-900c-fa3de271bdbb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:44.389104 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb565afe-9120-4e1e-900c-fa3de271bdbb service neutron] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:44.441983 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a762c8a4-fa35-4175-948b-5ff87798af2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:44.442590 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a762c8a4-fa35-4175-948b-5ff87798af2a service cinder] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:44.442867 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a762c8a4-fa35-4175-948b-5ff87798af2a service cinder] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:44.443137 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a762c8a4-fa35-4175-948b-5ff87798af2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:44.444916 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-a762c8a4-fa35-4175-948b-5ff87798af2a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:44.485982 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:51:44.509558 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a762c8a4-fa35-4175-948b-5ff87798af2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:44.511054 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a762c8a4-fa35-4175-948b-5ff87798af2a service cinder] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:44.521422 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1216/2443] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:51:44 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:51:48.058347 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c38e07-fe4b-4de8-989c-b275f5162277 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:51:48.060399 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-80c38e07-fe4b-4de8-989c-b275f5162277 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:48.074760 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:51:48.140370 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c38e07-fe4b-4de8-989c-b275f5162277 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:51:48.145819 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501f80f9-43f3-4ab3-847f-67218b7483fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bd5356829b6454993bab9bad2f79fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:51:48.146422 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501f80f9-43f3-4ab3-847f-67218b7483fd service nova] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:51:48.146701 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501f80f9-43f3-4ab3-847f-67218b7483fd service nova] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:51:48.146970 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501f80f9-43f3-4ab3-847f-67218b7483fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:51:48.148653 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-501f80f9-43f3-4ab3-847f-67218b7483fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:51:48.206541 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501f80f9-43f3-4ab3-847f-67218b7483fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:51:48.208233 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501f80f9-43f3-4ab3-847f-67218b7483fd service nova] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:51:48.219728 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1228/2444] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:51:48 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 10:51:54.517708 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=10 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:51:58.220154 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=50 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:52:04.506288 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a762c8a4-fa35-4175-948b-5ff87798af2a service cinder] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:52:04.507800 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-a762c8a4-fa35-4175-948b-5ff87798af2a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:52:04.569231 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a762c8a4-fa35-4175-948b-5ff87798af2a service cinder] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:52:04.572814 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19118527-8dcd-460f-8a8d-db1528d8e1b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:52:04.573412 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19118527-8dcd-460f-8a8d-db1528d8e1b7 service neutron] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:52:04.573690 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19118527-8dcd-460f-8a8d-db1528d8e1b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:52:04.573960 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19118527-8dcd-460f-8a8d-db1528d8e1b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:52:04.576278 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-19118527-8dcd-460f-8a8d-db1528d8e1b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:52:04.586707 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19118527-8dcd-460f-8a8d-db1528d8e1b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:52:04.588170 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19118527-8dcd-460f-8a8d-db1528d8e1b7 service neutron] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:52:04.597281 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1217/2445] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:52:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:52:07.661125 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58648b59-77bf-4d19-8ca4-bfb8d7eecd3a None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:52:07.661663 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58648b59-77bf-4d19-8ca4-bfb8d7eecd3a None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:52:07.661663 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58648b59-77bf-4d19-8ca4-bfb8d7eecd3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:52:07.685131 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-58648b59-77bf-4d19-8ca4-bfb8d7eecd3a None None] MFA Rules not processed for user `bb44e7aef0c7436d93a7233307b5f095`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:52:07.686399 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-58648b59-77bf-4d19-8ca4-bfb8d7eecd3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:52:07.759995 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1229/2446] 10.222.0.15 () {58 vars in 1009 bytes} [Sun Sep 11 10:52:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:52:07.782532 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19118527-8dcd-460f-8a8d-db1528d8e1b7 service neutron] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:52:07.784051 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-19118527-8dcd-460f-8a8d-db1528d8e1b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:52:07.853960 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19118527-8dcd-460f-8a8d-db1528d8e1b7 service neutron] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:52:07.859830 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283260df-7a29-4f68-9b24-54269c30a350 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7ca566e66a45b29e3d02f47bf84981', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:52:07.860430 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283260df-7a29-4f68-9b24-54269c30a350 service swift] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:52:07.860705 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283260df-7a29-4f68-9b24-54269c30a350 service swift] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:52:07.860974 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283260df-7a29-4f68-9b24-54269c30a350 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:52:07.869238 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283260df-7a29-4f68-9b24-54269c30a350 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:52:07.870819 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283260df-7a29-4f68-9b24-54269c30a350 service swift] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:52:07.880187 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1218/2447] 10.222.0.15 () {60 vars in 1384 bytes} [Sun Sep 11 10:52:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 10:52:17.744455 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=25 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:52:17.876505 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=58 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:52:24.247338 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58648b59-77bf-4d19-8ca4-bfb8d7eecd3a None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:52:24.256868 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58648b59-77bf-4d19-8ca4-bfb8d7eecd3a None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:52:24.260647 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462f3b72-c429-4cbf-9c00-fb136d5e5300 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:52:24.261332 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462f3b72-c429-4cbf-9c00-fb136d5e5300 service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:52:24.261620 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462f3b72-c429-4cbf-9c00-fb136d5e5300 service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:52:24.261891 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462f3b72-c429-4cbf-9c00-fb136d5e5300 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:52:24.263650 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-462f3b72-c429-4cbf-9c00-fb136d5e5300 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:52:24.274088 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462f3b72-c429-4cbf-9c00-fb136d5e5300 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:52:24.275589 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462f3b72-c429-4cbf-9c00-fb136d5e5300 service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:52:24.284198 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1230/2448] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:52:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:52:24.422842 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad82e35-b2be-43a7-a3c9-701bb7aa96a0 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:52:24.423740 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad82e35-b2be-43a7-a3c9-701bb7aa96a0 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:52:24.424441 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad82e35-b2be-43a7-a3c9-701bb7aa96a0 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:52:24.426066 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1219/2449] 10.222.0.15 () {54 vars in 921 bytes} [Sun Sep 11 10:52:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:52:24.437323 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9d0556-8479-45df-a581-58ef1847916b None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:52:24.437607 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9d0556-8479-45df-a581-58ef1847916b None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:52:24.438135 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9d0556-8479-45df-a581-58ef1847916b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:52:24.475661 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-bb9d0556-8479-45df-a581-58ef1847916b None None] MFA Rules not processed for user `2bd5356829b6454993bab9bad2f79fed`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:52:24.475661 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-bb9d0556-8479-45df-a581-58ef1847916b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:52:24.475661 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1231/2450] 10.222.0.15 () {58 vars in 1018 bytes} [Sun Sep 11 10:52:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:52:24.485484 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad82e35-b2be-43a7-a3c9-701bb7aa96a0 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:52:24.486921 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9d0556-8479-45df-a581-58ef1847916b None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:52:24.487699 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-fad82e35-b2be-43a7-a3c9-701bb7aa96a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:52:24.488416 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-bb9d0556-8479-45df-a581-58ef1847916b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:52:24.502158 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9d0556-8479-45df-a581-58ef1847916b None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:52:24.506383 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0004fdcd-03df-4e73-a9ab-fbc12d97d44b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:52:24.507014 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0004fdcd-03df-4e73-a9ab-fbc12d97d44b service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:52:24.507279 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0004fdcd-03df-4e73-a9ab-fbc12d97d44b service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:52:24.507601 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0004fdcd-03df-4e73-a9ab-fbc12d97d44b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:52:24.510225 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-0004fdcd-03df-4e73-a9ab-fbc12d97d44b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:52:24.560809 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad82e35-b2be-43a7-a3c9-701bb7aa96a0 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:52:24.565153 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d35e04-f4ce-4bd3-9aed-798024b9eb39 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1632da5fbe384b5c849be1d4d86c0f1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:52:24.565978 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d35e04-f4ce-4bd3-9aed-798024b9eb39 service glance] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:52:24.566326 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d35e04-f4ce-4bd3-9aed-798024b9eb39 service glance] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:52:24.566864 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d35e04-f4ce-4bd3-9aed-798024b9eb39 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:52:24.577690 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d35e04-f4ce-4bd3-9aed-798024b9eb39 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:52:24.579151 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d35e04-f4ce-4bd3-9aed-798024b9eb39 service glance] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:52:24.590644 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1220/2451] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:52:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:52:24.590825 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0004fdcd-03df-4e73-a9ab-fbc12d97d44b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:52:24.592163 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0004fdcd-03df-4e73-a9ab-fbc12d97d44b service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:52:24.603032 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1232/2452] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:52:24 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:52:24.663349 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429fafdc-974b-4fc3-839c-14cb30375e3d None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:52:24.663689 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429fafdc-974b-4fc3-839c-14cb30375e3d None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:52:24.663990 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429fafdc-974b-4fc3-839c-14cb30375e3d None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:52:24.665328 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1221/2453] 10.222.0.15 () {54 vars in 921 bytes} [Sun Sep 11 10:52:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:52:24.671386 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aac2ef-06db-4509-adb3-9e0017804fe1 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:52:24.671732 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aac2ef-06db-4509-adb3-9e0017804fe1 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:52:24.672035 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aac2ef-06db-4509-adb3-9e0017804fe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:52:24.693094 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-25aac2ef-06db-4509-adb3-9e0017804fe1 None None] MFA Rules not processed for user `40072bc8dcb5454382a93e20b63f8594`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:52:24.694299 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-25aac2ef-06db-4509-adb3-9e0017804fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:52:24.701009 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1233/2454] 10.222.0.15 () {58 vars in 1018 bytes} [Sun Sep 11 10:52:24 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:52:24.712936 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429fafdc-974b-4fc3-839c-14cb30375e3d None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:52:24.725131 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429fafdc-974b-4fc3-839c-14cb30375e3d None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:52:24.728818 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1df66a-0f73-448a-8f11-e658b453f701 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:52:24.729416 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1df66a-0f73-448a-8f11-e658b453f701 service neutron] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:52:24.729807 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1df66a-0f73-448a-8f11-e658b453f701 service neutron] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:52:24.730739 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1df66a-0f73-448a-8f11-e658b453f701 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:52:24.738354 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1df66a-0f73-448a-8f11-e658b453f701 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:52:24.739822 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1df66a-0f73-448a-8f11-e658b453f701 service neutron] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:52:24.748096 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1222/2455] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:52:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:52:24.836335 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2b7bd3-bb2c-44a5-9671-2bdff3414682 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:52:24.836625 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2b7bd3-bb2c-44a5-9671-2bdff3414682 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:52:24.836894 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2b7bd3-bb2c-44a5-9671-2bdff3414682 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:52:24.838279 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1234/2456] 10.222.0.15 () {54 vars in 921 bytes} [Sun Sep 11 10:52:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:52:24.844732 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa1d3f7-56ff-4072-b16c-84f972f5688f None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:52:24.845016 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa1d3f7-56ff-4072-b16c-84f972f5688f None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:52:24.845288 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa1d3f7-56ff-4072-b16c-84f972f5688f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:52:24.868892 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-2aa1d3f7-56ff-4072-b16c-84f972f5688f None None] MFA Rules not processed for user `2bd5356829b6454993bab9bad2f79fed`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:52:24.870118 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-2aa1d3f7-56ff-4072-b16c-84f972f5688f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:52:24.888458 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1223/2457] 10.222.0.15 () {58 vars in 1018 bytes} [Sun Sep 11 10:52:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 10:52:24.895148 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2b7bd3-bb2c-44a5-9671-2bdff3414682 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:52:24.908630 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2b7bd3-bb2c-44a5-9671-2bdff3414682 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:52:24.912439 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35611ff-3fb6-444c-bc6c-a6b78c55840d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:52:24.913046 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35611ff-3fb6-444c-bc6c-a6b78c55840d service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:52:24.913327 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35611ff-3fb6-444c-bc6c-a6b78c55840d service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:52:24.913596 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35611ff-3fb6-444c-bc6c-a6b78c55840d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:52:24.924863 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35611ff-3fb6-444c-bc6c-a6b78c55840d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:52:24.926309 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35611ff-3fb6-444c-bc6c-a6b78c55840d service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:52:24.938282 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1235/2458] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:52:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 10:52:25.007745 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa1d3f7-56ff-4072-b16c-84f972f5688f None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:52:25.009135 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-2aa1d3f7-56ff-4072-b16c-84f972f5688f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:52:25.082228 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa1d3f7-56ff-4072-b16c-84f972f5688f None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:52:25.087471 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe7e60b-8148-4a92-9080-d4e18de75a95 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08477b4c88f04c9e97391aba401879e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:52:25.088146 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe7e60b-8148-4a92-9080-d4e18de75a95 service placement] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:52:25.088548 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe7e60b-8148-4a92-9080-d4e18de75a95 service placement] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:52:25.088848 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe7e60b-8148-4a92-9080-d4e18de75a95 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:52:25.092106 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-7fe7e60b-8148-4a92-9080-d4e18de75a95 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:52:25.112481 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe7e60b-8148-4a92-9080-d4e18de75a95 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:52:25.114173 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe7e60b-8148-4a92-9080-d4e18de75a95 service placement] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:52:25.125010 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1224/2459] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:52:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:52:26.746736 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35611ff-3fb6-444c-bc6c-a6b78c55840d service neutron] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:52:26.762280 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35611ff-3fb6-444c-bc6c-a6b78c55840d service neutron] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:52:26.768549 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8905de-d8b8-4bb5-b14b-4fda876706eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bd5356829b6454993bab9bad2f79fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:52:26.769358 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8905de-d8b8-4bb5-b14b-4fda876706eb service nova] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:52:26.769886 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8905de-d8b8-4bb5-b14b-4fda876706eb service nova] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:52:26.770315 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8905de-d8b8-4bb5-b14b-4fda876706eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:52:26.772313 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-7a8905de-d8b8-4bb5-b14b-4fda876706eb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:52:26.786267 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8905de-d8b8-4bb5-b14b-4fda876706eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:52:26.788622 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8905de-d8b8-4bb5-b14b-4fda876706eb service nova] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:52:26.808987 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1236/2460] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:52:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:52:29.408173 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9180d61c-c132-498f-b43e-b7c81b1e325d None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:52:29.408736 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9180d61c-c132-498f-b43e-b7c81b1e325d None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:52:29.408736 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9180d61c-c132-498f-b43e-b7c81b1e325d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:52:29.429253 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-9180d61c-c132-498f-b43e-b7c81b1e325d None None] MFA Rules not processed for user `bb44e7aef0c7436d93a7233307b5f095`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:52:29.430471 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-9180d61c-c132-498f-b43e-b7c81b1e325d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:52:29.444029 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1225/2461] 10.222.0.15 () {58 vars in 1009 bytes} [Sun Sep 11 10:52:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:52:29.452373 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8905de-d8b8-4bb5-b14b-4fda876706eb service nova] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:52:29.465923 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8905de-d8b8-4bb5-b14b-4fda876706eb service nova] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:52:29.472265 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5142d9a-9a19-46eb-b407-a72a616a6531 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7ca566e66a45b29e3d02f47bf84981', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:52:29.473131 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5142d9a-9a19-46eb-b407-a72a616a6531 service swift] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:52:29.473405 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5142d9a-9a19-46eb-b407-a72a616a6531 service swift] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:52:29.473677 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5142d9a-9a19-46eb-b407-a72a616a6531 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:52:29.484068 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5142d9a-9a19-46eb-b407-a72a616a6531 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:52:29.485670 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5142d9a-9a19-46eb-b407-a72a616a6531 service swift] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:52:29.494740 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1237/2462] 10.222.0.15 () {60 vars in 1384 bytes} [Sun Sep 11 10:52:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 11 10:52:31.481049 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9180d61c-c132-498f-b43e-b7c81b1e325d None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:52:31.482690 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-9180d61c-c132-498f-b43e-b7c81b1e325d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:52:31.500414 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9180d61c-c132-498f-b43e-b7c81b1e325d None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:52:31.505980 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e431440-07ce-472d-9194-1a78f3d11321 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bd5356829b6454993bab9bad2f79fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:52:31.506612 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e431440-07ce-472d-9194-1a78f3d11321 service nova] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:52:31.506894 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e431440-07ce-472d-9194-1a78f3d11321 service nova] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:52:31.507168 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e431440-07ce-472d-9194-1a78f3d11321 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:52:31.508911 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-7e431440-07ce-472d-9194-1a78f3d11321 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:52:31.570165 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e431440-07ce-472d-9194-1a78f3d11321 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:52:31.571758 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e431440-07ce-472d-9194-1a78f3d11321 service nova] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:52:31.581954 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1226/2463] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:52:31 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:52:39.491290 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=60 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:52:41.577373 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=98 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:52:56.581777 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5142d9a-9a19-46eb-b407-a72a616a6531 service swift] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:52:56.595901 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5142d9a-9a19-46eb-b407-a72a616a6531 service swift] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:52:56.601475 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f80d97-f007-4584-a98f-2533f29d084c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bd5356829b6454993bab9bad2f79fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:52:56.602075 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f80d97-f007-4584-a98f-2533f29d084c service nova] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:52:56.602352 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f80d97-f007-4584-a98f-2533f29d084c service nova] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:52:56.602621 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f80d97-f007-4584-a98f-2533f29d084c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:52:56.604386 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-09f80d97-f007-4584-a98f-2533f29d084c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:52:56.616609 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f80d97-f007-4584-a98f-2533f29d084c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:52:56.618170 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f80d97-f007-4584-a98f-2533f29d084c service nova] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:52:56.628736 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1238/2464] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:52:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:53:00.410628 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78205f30-2c20-4294-8270-cad38485b986 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:53:00.411078 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78205f30-2c20-4294-8270-cad38485b986 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:53:00.411171 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78205f30-2c20-4294-8270-cad38485b986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:53:00.431148 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-78205f30-2c20-4294-8270-cad38485b986 None None] MFA Rules not processed for user `bb44e7aef0c7436d93a7233307b5f095`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:53:00.432386 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-78205f30-2c20-4294-8270-cad38485b986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:53:00.440221 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1227/2465] 10.222.0.15 () {58 vars in 967 bytes} [Sun Sep 11 10:53:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:53:00.452765 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f80d97-f007-4584-a98f-2533f29d084c service nova] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:53:00.463789 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f80d97-f007-4584-a98f-2533f29d084c service nova] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:53:00.468945 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f02cd9-e133-47bc-a535-8322f268db69 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7ca566e66a45b29e3d02f47bf84981', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:53:00.469540 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f02cd9-e133-47bc-a535-8322f268db69 service swift] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:53:00.469850 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f02cd9-e133-47bc-a535-8322f268db69 service swift] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:53:00.470114 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f02cd9-e133-47bc-a535-8322f268db69 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:53:00.478132 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f02cd9-e133-47bc-a535-8322f268db69 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:53:00.479739 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f02cd9-e133-47bc-a535-8322f268db69 service swift] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:53:00.488606 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1239/2466] 10.222.0.15 () {60 vars in 1384 bytes} [Sun Sep 11 10:53:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 10:53:10.425537 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=3 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:53:10.485313 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=12 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:53:16.112271 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb423e6-9289-4779-a9e4-c9c9edc2d26c None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:53:16.112822 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb423e6-9289-4779-a9e4-c9c9edc2d26c None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:53:16.112822 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb423e6-9289-4779-a9e4-c9c9edc2d26c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:53:16.131851 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-ffb423e6-9289-4779-a9e4-c9c9edc2d26c None None] MFA Rules not processed for user `f6eaba3a278e4bc99aca0dedd649fa9f`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:53:16.133027 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-ffb423e6-9289-4779-a9e4-c9c9edc2d26c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:53:16.140160 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1228/2467] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:53:16 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:53:21.321347 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f02cd9-e133-47bc-a535-8322f268db69 service swift] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:53:21.334794 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f02cd9-e133-47bc-a535-8322f268db69 service swift] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:53:21.340665 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e76c11-cdcd-4b4e-9a7b-22cb4dde43a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bd5356829b6454993bab9bad2f79fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:53:21.341289 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e76c11-cdcd-4b4e-9a7b-22cb4dde43a4 service nova] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:53:21.341560 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e76c11-cdcd-4b4e-9a7b-22cb4dde43a4 service nova] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:53:21.341823 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e76c11-cdcd-4b4e-9a7b-22cb4dde43a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:53:21.343604 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-44e76c11-cdcd-4b4e-9a7b-22cb4dde43a4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:53:21.356450 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e76c11-cdcd-4b4e-9a7b-22cb4dde43a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:53:21.358041 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e76c11-cdcd-4b4e-9a7b-22cb4dde43a4 service nova] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:53:21.368960 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1240/2468] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:53:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:53:26.126605 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=3 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:53:31.364436 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=6 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:54:03.680043 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78cf1db-0110-4253-9498-6338ae88e446 None None] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:54:03.680622 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78cf1db-0110-4253-9498-6338ae88e446 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:54:03.680756 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78cf1db-0110-4253-9498-6338ae88e446 None None] PATH_INFO: `/` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:54:03.682746 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1229/2469] 10.222.0.15 () {54 vars in 921 bytes} [Sun Sep 11 10:54:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:54:03.688818 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb42752-0910-4b8b-9b91-1c4232038b23 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:54:03.689373 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb42752-0910-4b8b-9b91-1c4232038b23 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:54:03.689543 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb42752-0910-4b8b-9b91-1c4232038b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:54:03.709383 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-6eb42752-0910-4b8b-9b91-1c4232038b23 None None] MFA Rules not processed for user `40072bc8dcb5454382a93e20b63f8594`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:54:03.710587 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-6eb42752-0910-4b8b-9b91-1c4232038b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:54:03.717871 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1241/2470] 10.222.0.15 () {58 vars in 1018 bytes} [Sun Sep 11 10:54:03 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:54:03.730644 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78cf1db-0110-4253-9498-6338ae88e446 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:54:03.741987 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78cf1db-0110-4253-9498-6338ae88e446 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:54:03.745675 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd69580d-7f9a-4543-afb5-6e44ee369b93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:54:03.746277 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd69580d-7f9a-4543-afb5-6e44ee369b93 service neutron] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:54:03.746557 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd69580d-7f9a-4543-afb5-6e44ee369b93 service neutron] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:54:03.746834 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd69580d-7f9a-4543-afb5-6e44ee369b93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:54:03.756006 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd69580d-7f9a-4543-afb5-6e44ee369b93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:54:03.757433 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd69580d-7f9a-4543-afb5-6e44ee369b93 service neutron] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:54:03.766115 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1230/2471] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:54:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:54:03.858188 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c95ff16-2c03-4210-b8aa-0d7a1318d8c9 None None] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:54:03.858963 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c95ff16-2c03-4210-b8aa-0d7a1318d8c9 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:54:03.859246 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c95ff16-2c03-4210-b8aa-0d7a1318d8c9 None None] PATH_INFO: `/` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:54:03.860622 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1242/2472] 10.222.0.15 () {54 vars in 921 bytes} [Sun Sep 11 10:54:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 10:54:03.868074 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6177ed2a-be89-4368-984d-f79871eddba9 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:54:03.868614 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6177ed2a-be89-4368-984d-f79871eddba9 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:54:03.868908 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6177ed2a-be89-4368-984d-f79871eddba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:54:03.890179 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-6177ed2a-be89-4368-984d-f79871eddba9 None None] MFA Rules not processed for user `2bd5356829b6454993bab9bad2f79fed`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:54:03.891442 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-6177ed2a-be89-4368-984d-f79871eddba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:54:03.903579 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1231/2473] 10.222.0.15 () {58 vars in 1018 bytes} [Sun Sep 11 10:54:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:54:03.920493 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c95ff16-2c03-4210-b8aa-0d7a1318d8c9 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:54:03.933133 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c95ff16-2c03-4210-b8aa-0d7a1318d8c9 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:54:03.936695 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dd9250-43a4-4310-b40c-0dfd0a18a29e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:54:03.937322 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dd9250-43a4-4310-b40c-0dfd0a18a29e service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:54:03.937597 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dd9250-43a4-4310-b40c-0dfd0a18a29e service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:54:03.937860 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dd9250-43a4-4310-b40c-0dfd0a18a29e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:54:03.948825 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dd9250-43a4-4310-b40c-0dfd0a18a29e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:54:03.950249 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dd9250-43a4-4310-b40c-0dfd0a18a29e service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:54:03.961518 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1243/2474] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:54:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:54:13.884892 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=9 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:54:13.956896 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=9 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:54:20.412989 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3744d1ec-a44f-40bc-afbd-d112debe413b None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:54:20.413472 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3744d1ec-a44f-40bc-afbd-d112debe413b None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:54:20.413580 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3744d1ec-a44f-40bc-afbd-d112debe413b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:54:20.435406 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-3744d1ec-a44f-40bc-afbd-d112debe413b None None] MFA Rules not processed for user `e1b0815853b947bc8f50d91c1ce0ab5a`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:54:20.436777 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-3744d1ec-a44f-40bc-afbd-d112debe413b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:54:20.512043 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1232/2475] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:54:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:54:22.614929 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ec4659-0920-4822-82a5-b4a1635176de None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:54:22.615535 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ec4659-0920-4822-82a5-b4a1635176de None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:54:22.615535 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ec4659-0920-4822-82a5-b4a1635176de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:54:22.636253 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-c6ec4659-0920-4822-82a5-b4a1635176de None None] MFA Rules not processed for user `2238a62cbf1846999b31ea9434b67743`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:54:22.637499 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-c6ec4659-0920-4822-82a5-b4a1635176de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:54:22.644634 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1244/2476] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:54:22 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:54:30.501937 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=25 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:54:32.630615 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=3 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:54:48.689698 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3744d1ec-a44f-40bc-afbd-d112debe413b None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:54:48.750895 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3744d1ec-a44f-40bc-afbd-d112debe413b None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:54:48.755946 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f7c1e3-6ce8-4bca-915d-c94697267a4e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08477b4c88f04c9e97391aba401879e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:54:48.756545 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f7c1e3-6ce8-4bca-915d-c94697267a4e service placement] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:54:48.756822 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f7c1e3-6ce8-4bca-915d-c94697267a4e service placement] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:54:48.757090 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f7c1e3-6ce8-4bca-915d-c94697267a4e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:54:48.759772 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-a2f7c1e3-6ce8-4bca-915d-c94697267a4e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:54:48.793883 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f7c1e3-6ce8-4bca-915d-c94697267a4e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:54:48.795497 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f7c1e3-6ce8-4bca-915d-c94697267a4e service placement] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:54:48.806443 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1233/2477] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:54:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:54:53.203140 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ec4659-0920-4822-82a5-b4a1635176de None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:54:53.216457 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ec4659-0920-4822-82a5-b4a1635176de None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:54:53.219995 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4912bbe-010f-4e64-ad35-0ebbb298153c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:54:53.220588 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4912bbe-010f-4e64-ad35-0ebbb298153c service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:54:53.220865 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4912bbe-010f-4e64-ad35-0ebbb298153c service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:54:53.221132 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4912bbe-010f-4e64-ad35-0ebbb298153c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:54:53.222793 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-d4912bbe-010f-4e64-ad35-0ebbb298153c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:54:53.307088 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4912bbe-010f-4e64-ad35-0ebbb298153c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:54:53.308983 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4912bbe-010f-4e64-ad35-0ebbb298153c service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:54:53.328495 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1245/2478] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:54:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:54:56.128406 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f7c1e3-6ce8-4bca-915d-c94697267a4e service placement] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:54:56.130030 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-a2f7c1e3-6ce8-4bca-915d-c94697267a4e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:54:56.144356 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f7c1e3-6ce8-4bca-915d-c94697267a4e service placement] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:54:56.149681 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2122ece-18da-4c91-bda0-24ad1e1e259b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08477b4c88f04c9e97391aba401879e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:54:56.150958 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2122ece-18da-4c91-bda0-24ad1e1e259b service placement] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:54:56.151244 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2122ece-18da-4c91-bda0-24ad1e1e259b service placement] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:54:56.151579 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2122ece-18da-4c91-bda0-24ad1e1e259b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:54:56.153650 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-e2122ece-18da-4c91-bda0-24ad1e1e259b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:54:56.167053 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2122ece-18da-4c91-bda0-24ad1e1e259b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:54:56.168755 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2122ece-18da-4c91-bda0-24ad1e1e259b service placement] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:54:56.179678 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1234/2479] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:54:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:54:56.184855 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4912bbe-010f-4e64-ad35-0ebbb298153c service neutron] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:54:56.193054 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-d4912bbe-010f-4e64-ad35-0ebbb298153c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:54:56.212420 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4912bbe-010f-4e64-ad35-0ebbb298153c service neutron] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:54:56.217928 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1f28cc-8a8f-48eb-b2f0-19adf2707aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:54:56.218565 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1f28cc-8a8f-48eb-b2f0-19adf2707aa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:54:56.218845 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1f28cc-8a8f-48eb-b2f0-19adf2707aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:54:56.219118 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1f28cc-8a8f-48eb-b2f0-19adf2707aa3 admin admin] PATH_INFO: `/v3/users/f6eaba3a278e4bc99aca0dedd649fa9f` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:54:56.220322 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1f28cc-8a8f-48eb-b2f0-19adf2707aa3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6eaba3a278e4bc99aca0dedd649fa9f)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:54:56.222227 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1f28cc-8a8f-48eb-b2f0-19adf2707aa3 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:54:56.271790 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-2f1f28cc-8a8f-48eb-b2f0-19adf2707aa3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6eaba3a278e4bc99aca0dedd649fa9f'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:54:56.280940 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-2f1f28cc-8a8f-48eb-b2f0-19adf2707aa3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6eaba3a278e4bc99aca0dedd649fa9f'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:54:56.281647 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-2f1f28cc-8a8f-48eb-b2f0-19adf2707aa3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6eaba3a278e4bc99aca0dedd649fa9f'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:54:56.282320 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-2f1f28cc-8a8f-48eb-b2f0-19adf2707aa3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6eaba3a278e4bc99aca0dedd649fa9f'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:54:56.296695 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-2f1f28cc-8a8f-48eb-b2f0-19adf2707aa3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6eaba3a278e4bc99aca0dedd649fa9f'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:54:56.314091 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1246/2480] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:54:56 2022] DELETE /identity/v3/users/f6eaba3a278e4bc99aca0dedd649fa9f => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:54:56.484557 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2122ece-18da-4c91-bda0-24ad1e1e259b service placement] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:54:56.501240 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2122ece-18da-4c91-bda0-24ad1e1e259b service placement] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:54:56.560034 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9d5f72-20b2-47f8-b0fc-06744ee5847a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:54:56.560714 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9d5f72-20b2-47f8-b0fc-06744ee5847a admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:54:56.560950 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9d5f72-20b2-47f8-b0fc-06744ee5847a admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:54:56.561232 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9d5f72-20b2-47f8-b0fc-06744ee5847a admin admin] PATH_INFO: `/v3/projects/528ee82fc48247eea01a17898488ea3e` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:54:56.562336 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9d5f72-20b2-47f8-b0fc-06744ee5847a admin admin] RBAC: Authorizing `identity:delete_project(project_id=528ee82fc48247eea01a17898488ea3e)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:54:56.564270 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9d5f72-20b2-47f8-b0fc-06744ee5847a admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:54:56.609716 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-ab9d5f72-20b2-47f8-b0fc-06744ee5847a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '528ee82fc48247eea01a17898488ea3e', 'request_id': 'req-ab9d5f72-20b2-47f8-b0fc-06744ee5847a'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:54:56.615999 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-ab9d5f72-20b2-47f8-b0fc-06744ee5847a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '528ee82fc48247eea01a17898488ea3e', 'request_id': 'req-ab9d5f72-20b2-47f8-b0fc-06744ee5847a'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:54:56.624248 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-ab9d5f72-20b2-47f8-b0fc-06744ee5847a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '528ee82fc48247eea01a17898488ea3e', 'request_id': 'req-ab9d5f72-20b2-47f8-b0fc-06744ee5847a'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:54:56.630487 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:54:56.636099 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-ab9d5f72-20b2-47f8-b0fc-06744ee5847a admin admin] The token cache is being invalidate because project 528ee82fc48247eea01a17898488ea3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90653) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:54:56.636599 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-ab9d5f72-20b2-47f8-b0fc-06744ee5847a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:54:56.637642 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-ab9d5f72-20b2-47f8-b0fc-06744ee5847a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:54:56.639500 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1235/2481] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:54:56 2022] DELETE /identity/v3/projects/528ee82fc48247eea01a17898488ea3e => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:54:56.644434 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1efab-bd4f-4d75-9cf2-4fcabdfa67f1 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:54:56.644717 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1efab-bd4f-4d75-9cf2-4fcabdfa67f1 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:54:56.644988 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1efab-bd4f-4d75-9cf2-4fcabdfa67f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:54:56.694330 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:54:56.704977 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-35e1efab-bd4f-4d75-9cf2-4fcabdfa67f1 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:54:56.706417 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-35e1efab-bd4f-4d75-9cf2-4fcabdfa67f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:54:56.764072 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1247/2482] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:54:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 10:54:56.772460 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053fdda7-fb5f-46c2-95f9-ec84f3ece711 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:54:56.772748 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053fdda7-fb5f-46c2-95f9-ec84f3ece711 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:54:56.773020 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053fdda7-fb5f-46c2-95f9-ec84f3ece711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:54:56.793031 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-053fdda7-fb5f-46c2-95f9-ec84f3ece711 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:54:56.794214 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-053fdda7-fb5f-46c2-95f9-ec84f3ece711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:54:56.801959 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1236/2483] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:54:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:54:56.807314 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e1efab-bd4f-4d75-9cf2-4fcabdfa67f1 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:54:56.817972 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e1efab-bd4f-4d75-9cf2-4fcabdfa67f1 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:54:56.822611 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe19760-f35b-4811-b98b-b9eadad490a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:54:56.823200 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe19760-f35b-4811-b98b-b9eadad490a1 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:54:56.823536 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe19760-f35b-4811-b98b-b9eadad490a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:54:56.823809 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe19760-f35b-4811-b98b-b9eadad490a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:54:56.824442 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe19760-f35b-4811-b98b-b9eadad490a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:54:56.824742 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe19760-f35b-4811-b98b-b9eadad490a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:54:56.826321 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe19760-f35b-4811-b98b-b9eadad490a1 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:54:56.837473 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1248/2484] 10.222.0.15 () {58 vars in 1153 bytes} [Sun Sep 11 10:54:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:54:56.841794 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053fdda7-fb5f-46c2-95f9-ec84f3ece711 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:54:56.850817 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053fdda7-fb5f-46c2-95f9-ec84f3ece711 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:54:56.856302 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979e4907-af21-48e1-8d6d-c8ed7000cf35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:54:56.856898 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979e4907-af21-48e1-8d6d-c8ed7000cf35 admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:54:56.857180 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979e4907-af21-48e1-8d6d-c8ed7000cf35 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:54:56.857450 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979e4907-af21-48e1-8d6d-c8ed7000cf35 admin admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:54:56.858087 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979e4907-af21-48e1-8d6d-c8ed7000cf35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:54:56.860319 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979e4907-af21-48e1-8d6d-c8ed7000cf35 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:54:56.888212 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1237/2485] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:54:56 2022] POST /identity/v3/projects => generated 405 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:54:56.892659 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe19760-f35b-4811-b98b-b9eadad490a1 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:54:56.903329 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe19760-f35b-4811-b98b-b9eadad490a1 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:54:56.965024 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e4b36e-e92d-4ff3-9d7a-9edd37464200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:54:56.965629 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e4b36e-e92d-4ff3-9d7a-9edd37464200 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:54:56.965911 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e4b36e-e92d-4ff3-9d7a-9edd37464200 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:54:56.966188 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e4b36e-e92d-4ff3-9d7a-9edd37464200 admin admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:54:56.966808 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e4b36e-e92d-4ff3-9d7a-9edd37464200 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:54:56.968789 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e4b36e-e92d-4ff3-9d7a-9edd37464200 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:54:56.989816 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1249/2486] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:54:56 2022] POST /identity/v3/users => generated 303 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:54:56.994733 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979e4907-af21-48e1-8d6d-c8ed7000cf35 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:54:57.005079 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979e4907-af21-48e1-8d6d-c8ed7000cf35 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:54:57.010318 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275aff92-460e-480e-a981-ce7e55a82354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:54:57.010907 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275aff92-460e-480e-a981-ce7e55a82354 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:54:57.011186 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275aff92-460e-480e-a981-ce7e55a82354 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:54:57.011487 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275aff92-460e-480e-a981-ce7e55a82354 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:54:57.012119 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275aff92-460e-480e-a981-ce7e55a82354 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:54:57.013704 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275aff92-460e-480e-a981-ce7e55a82354 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:54:57.023487 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1238/2487] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:54:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:54:57.027535 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e4b36e-e92d-4ff3-9d7a-9edd37464200 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:54:57.037323 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e4b36e-e92d-4ff3-9d7a-9edd37464200 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:54:57.042100 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb58f829-bd7b-4dcf-a88b-7065b27de18f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:54:57.042687 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb58f829-bd7b-4dcf-a88b-7065b27de18f admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:54:57.042969 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb58f829-bd7b-4dcf-a88b-7065b27de18f admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:54:57.043239 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb58f829-bd7b-4dcf-a88b-7065b27de18f admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:54:57.043906 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb58f829-bd7b-4dcf-a88b-7065b27de18f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:54:57.045529 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb58f829-bd7b-4dcf-a88b-7065b27de18f admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:54:57.055909 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1250/2488] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:54:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:54:57.060608 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275aff92-460e-480e-a981-ce7e55a82354 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:54:57.070590 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275aff92-460e-480e-a981-ce7e55a82354 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:54:57.075978 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06e4775-cc45-441f-b72d-adf3fc9000e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:54:57.076623 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e4775-cc45-441f-b72d-adf3fc9000e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:54:57.076905 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e4775-cc45-441f-b72d-adf3fc9000e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:54:57.077182 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e4775-cc45-441f-b72d-adf3fc9000e9 admin admin] PATH_INFO: `/v3/projects/f26071f798cf4e1890b9a20c2317d901/users/e8d8857fd433456f91938af99eda3561/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:54:57.091732 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06e4775-cc45-441f-b72d-adf3fc9000e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f26071f798cf4e1890b9a20c2317d901, user_id=e8d8857fd433456f91938af99eda3561, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:54:57.094351 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06e4775-cc45-441f-b72d-adf3fc9000e9 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:54:57.107018 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1239/2489] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:54:57 2022] PUT /identity/v3/projects/f26071f798cf4e1890b9a20c2317d901/users/e8d8857fd433456f91938af99eda3561/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 10:54:57.112298 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e491faf-62b9-4d92-a664-bcc4b49fee00 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:54:57.112530 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e491faf-62b9-4d92-a664-bcc4b49fee00 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:54:57.112807 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e491faf-62b9-4d92-a664-bcc4b49fee00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:54:57.132310 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-5e491faf-62b9-4d92-a664-bcc4b49fee00 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:54:57.133522 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-5e491faf-62b9-4d92-a664-bcc4b49fee00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:54:57.191974 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1251/2490] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:54:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:54:57.202225 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06e4775-cc45-441f-b72d-adf3fc9000e9 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:54:57.203947 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-e06e4775-cc45-441f-b72d-adf3fc9000e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:54:57.283472 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06e4775-cc45-441f-b72d-adf3fc9000e9 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:54:57.287060 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5a618c-005c-4502-9dab-4f430b4d7ae0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:54:57.287697 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a618c-005c-4502-9dab-4f430b4d7ae0 service neutron] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:54:57.287977 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a618c-005c-4502-9dab-4f430b4d7ae0 service neutron] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:54:57.288247 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a618c-005c-4502-9dab-4f430b4d7ae0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:54:57.298065 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5a618c-005c-4502-9dab-4f430b4d7ae0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:54:57.299510 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5a618c-005c-4502-9dab-4f430b4d7ae0 service neutron] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:54:57.309810 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1240/2491] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:54:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:54:57.913148 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcd968b-f71d-4efd-bd4a-7471cff6595d None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:54:57.913447 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcd968b-f71d-4efd-bd4a-7471cff6595d None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:54:57.913713 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcd968b-f71d-4efd-bd4a-7471cff6595d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:54:57.934273 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:54:57.944276 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-2dcd968b-f71d-4efd-bd4a-7471cff6595d None None] MFA Rules not processed for user `e8d8857fd433456f91938af99eda3561`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:54:57.946106 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-2dcd968b-f71d-4efd-bd4a-7471cff6595d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:54:58.016417 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1252/2492] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:54:57 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:54:58.020942 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5a618c-005c-4502-9dab-4f430b4d7ae0 service neutron] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:54:58.031775 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5a618c-005c-4502-9dab-4f430b4d7ae0 service neutron] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:54:58.036608 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b8d6da-c4a1-4a04-af38-bbbc66ae8fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:54:58.037232 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b8d6da-c4a1-4a04-af38-bbbc66ae8fc5 admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:54:58.037510 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b8d6da-c4a1-4a04-af38-bbbc66ae8fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:54:58.037780 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b8d6da-c4a1-4a04-af38-bbbc66ae8fc5 admin admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:54:58.038400 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b8d6da-c4a1-4a04-af38-bbbc66ae8fc5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:54:58.040324 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b8d6da-c4a1-4a04-af38-bbbc66ae8fc5 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:54:58.072355 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1241/2493] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:54:58 2022] POST /identity/v3/projects => generated 405 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 10:54:58.073804 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcd968b-f71d-4efd-bd4a-7471cff6595d None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:54:58.084573 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcd968b-f71d-4efd-bd4a-7471cff6595d None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:54:58.142340 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c452bbc8-0dbe-4362-ac3e-6bb3e16c9076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:54:58.142942 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c452bbc8-0dbe-4362-ac3e-6bb3e16c9076 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:54:58.143220 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c452bbc8-0dbe-4362-ac3e-6bb3e16c9076 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:54:58.143529 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c452bbc8-0dbe-4362-ac3e-6bb3e16c9076 admin admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:54:58.144147 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c452bbc8-0dbe-4362-ac3e-6bb3e16c9076 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:54:58.146088 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c452bbc8-0dbe-4362-ac3e-6bb3e16c9076 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:54:58.167900 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1253/2494] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:54:58 2022] POST /identity/v3/users => generated 309 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:54:58.171919 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b8d6da-c4a1-4a04-af38-bbbc66ae8fc5 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:54:58.184132 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b8d6da-c4a1-4a04-af38-bbbc66ae8fc5 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:54:58.188790 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6943c0-f41c-4b8f-9d0c-2af49e286621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:54:58.189362 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6943c0-f41c-4b8f-9d0c-2af49e286621 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:54:58.189642 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6943c0-f41c-4b8f-9d0c-2af49e286621 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:54:58.189912 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6943c0-f41c-4b8f-9d0c-2af49e286621 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:54:58.190575 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6943c0-f41c-4b8f-9d0c-2af49e286621 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:54:58.192228 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6943c0-f41c-4b8f-9d0c-2af49e286621 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:54:58.202668 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1242/2495] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:54:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:54:58.206693 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c452bbc8-0dbe-4362-ac3e-6bb3e16c9076 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:54:58.217223 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c452bbc8-0dbe-4362-ac3e-6bb3e16c9076 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:54:58.221760 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d040f4-60b4-45df-b96b-d718e06a9b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:54:58.222395 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d040f4-60b4-45df-b96b-d718e06a9b3b admin admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:54:58.222671 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d040f4-60b4-45df-b96b-d718e06a9b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:54:58.222936 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d040f4-60b4-45df-b96b-d718e06a9b3b admin admin] PATH_INFO: `/v3/projects/9d0bbffeca494c07a340e40ee2a5c42b/users/59b2fdc636d04757a8a942111b6cfe3d/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:54:58.237701 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d040f4-60b4-45df-b96b-d718e06a9b3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d0bbffeca494c07a340e40ee2a5c42b, user_id=59b2fdc636d04757a8a942111b6cfe3d, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:54:58.240364 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d040f4-60b4-45df-b96b-d718e06a9b3b admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:54:58.253011 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1254/2496] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:54:58 2022] PUT /identity/v3/projects/9d0bbffeca494c07a340e40ee2a5c42b/users/59b2fdc636d04757a8a942111b6cfe3d/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:54:58.256826 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6943c0-f41c-4b8f-9d0c-2af49e286621 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:54:58.267531 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6943c0-f41c-4b8f-9d0c-2af49e286621 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:54:58.312274 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:54:58.329155 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad7c0e6-54c7-4e18-8c85-da8945ac10ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:54:58.329754 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad7c0e6-54c7-4e18-8c85-da8945ac10ff admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:54:58.330051 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad7c0e6-54c7-4e18-8c85-da8945ac10ff admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:54:58.330316 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad7c0e6-54c7-4e18-8c85-da8945ac10ff admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:54:58.330942 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad7c0e6-54c7-4e18-8c85-da8945ac10ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:54:58.332552 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad7c0e6-54c7-4e18-8c85-da8945ac10ff admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:54:58.342444 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1243/2497] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:54:58 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:54:58.346632 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d040f4-60b4-45df-b96b-d718e06a9b3b admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:54:58.356893 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d040f4-60b4-45df-b96b-d718e06a9b3b admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:54:58.361765 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37225dbc-0396-4195-91e8-7dde4f039f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:54:58.362410 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37225dbc-0396-4195-91e8-7dde4f039f76 admin admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:54:58.362690 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37225dbc-0396-4195-91e8-7dde4f039f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:54:58.362964 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37225dbc-0396-4195-91e8-7dde4f039f76 admin admin] PATH_INFO: `/v3/projects/9d0bbffeca494c07a340e40ee2a5c42b/users/59b2fdc636d04757a8a942111b6cfe3d/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:54:58.364906 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37225dbc-0396-4195-91e8-7dde4f039f76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d0bbffeca494c07a340e40ee2a5c42b, user_id=59b2fdc636d04757a8a942111b6cfe3d, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:54:58.367586 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37225dbc-0396-4195-91e8-7dde4f039f76 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:54:58.377597 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1255/2498] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:54:58 2022] PUT /identity/v3/projects/9d0bbffeca494c07a340e40ee2a5c42b/users/59b2fdc636d04757a8a942111b6cfe3d/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:54:58.998922 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e13fb2-7798-4128-979d-01610405c71d None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:54:58.999329 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e13fb2-7798-4128-979d-01610405c71d None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:54:58.999648 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e13fb2-7798-4128-979d-01610405c71d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:54:59.027775 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-d5e13fb2-7798-4128-979d-01610405c71d None None] MFA Rules not processed for user `59b2fdc636d04757a8a942111b6cfe3d`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:54:59.029000 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-d5e13fb2-7798-4128-979d-01610405c71d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:54:59.101751 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1244/2499] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:54:58 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:55:00.316053 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37225dbc-0396-4195-91e8-7dde4f039f76 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:00.367705 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:55:00.379659 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37225dbc-0396-4195-91e8-7dde4f039f76 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:00.385079 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bade4262-b95e-488c-9d6f-0b2f6be3bba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:00.385816 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade4262-b95e-488c-9d6f-0b2f6be3bba3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:00.386093 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade4262-b95e-488c-9d6f-0b2f6be3bba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:00.386362 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade4262-b95e-488c-9d6f-0b2f6be3bba3 admin admin] PATH_INFO: `/v3/users/e8d8857fd433456f91938af99eda3561` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:00.387546 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bade4262-b95e-488c-9d6f-0b2f6be3bba3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8d8857fd433456f91938af99eda3561)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:00.389448 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bade4262-b95e-488c-9d6f-0b2f6be3bba3 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:00.436604 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-bade4262-b95e-488c-9d6f-0b2f6be3bba3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8d8857fd433456f91938af99eda3561'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:00.444968 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-bade4262-b95e-488c-9d6f-0b2f6be3bba3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8d8857fd433456f91938af99eda3561'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:00.445606 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-bade4262-b95e-488c-9d6f-0b2f6be3bba3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8d8857fd433456f91938af99eda3561'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:00.446217 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-bade4262-b95e-488c-9d6f-0b2f6be3bba3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8d8857fd433456f91938af99eda3561'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:00.459371 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-bade4262-b95e-488c-9d6f-0b2f6be3bba3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8d8857fd433456f91938af99eda3561'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:00.472288 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1256/2500] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:55:00 2022] DELETE /identity/v3/users/e8d8857fd433456f91938af99eda3561 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:55:00.640227 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e13fb2-7798-4128-979d-01610405c71d None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:00.651467 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e13fb2-7798-4128-979d-01610405c71d None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:00.708804 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec087d5b-9163-42fd-be55-412d694a42ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:00.709364 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec087d5b-9163-42fd-be55-412d694a42ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:00.709642 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec087d5b-9163-42fd-be55-412d694a42ae admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:00.709916 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec087d5b-9163-42fd-be55-412d694a42ae admin admin] PATH_INFO: `/v3/projects/f26071f798cf4e1890b9a20c2317d901` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:00.711035 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec087d5b-9163-42fd-be55-412d694a42ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=f26071f798cf4e1890b9a20c2317d901)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:00.712997 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec087d5b-9163-42fd-be55-412d694a42ae admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:00.765374 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-ec087d5b-9163-42fd-be55-412d694a42ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f26071f798cf4e1890b9a20c2317d901', 'request_id': 'req-ec087d5b-9163-42fd-be55-412d694a42ae'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:00.771804 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-ec087d5b-9163-42fd-be55-412d694a42ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f26071f798cf4e1890b9a20c2317d901', 'request_id': 'req-ec087d5b-9163-42fd-be55-412d694a42ae'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:00.779607 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-ec087d5b-9163-42fd-be55-412d694a42ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f26071f798cf4e1890b9a20c2317d901', 'request_id': 'req-ec087d5b-9163-42fd-be55-412d694a42ae'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:00.784553 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:55:00.792008 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-ec087d5b-9163-42fd-be55-412d694a42ae admin admin] The token cache is being invalidate because project f26071f798cf4e1890b9a20c2317d901 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90653) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:55:00.792534 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-ec087d5b-9163-42fd-be55-412d694a42ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:00.793470 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-ec087d5b-9163-42fd-be55-412d694a42ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:00.794988 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1245/2501] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:55:00 2022] DELETE /identity/v3/projects/f26071f798cf4e1890b9a20c2317d901 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:55:00.798970 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bade4262-b95e-488c-9d6f-0b2f6be3bba3 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:00.800510 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-bade4262-b95e-488c-9d6f-0b2f6be3bba3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:55:00.814159 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bade4262-b95e-488c-9d6f-0b2f6be3bba3 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:00.870292 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558021f8-076f-4b14-afd2-4bf973ae2435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:00.870925 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558021f8-076f-4b14-afd2-4bf973ae2435 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:00.871202 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558021f8-076f-4b14-afd2-4bf973ae2435 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:00.871500 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558021f8-076f-4b14-afd2-4bf973ae2435 admin admin] PATH_INFO: `/v3/users/59b2fdc636d04757a8a942111b6cfe3d` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:00.872625 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558021f8-076f-4b14-afd2-4bf973ae2435 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59b2fdc636d04757a8a942111b6cfe3d)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:00.874536 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558021f8-076f-4b14-afd2-4bf973ae2435 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:00.919978 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-558021f8-076f-4b14-afd2-4bf973ae2435 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59b2fdc636d04757a8a942111b6cfe3d'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:00.927074 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-558021f8-076f-4b14-afd2-4bf973ae2435 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59b2fdc636d04757a8a942111b6cfe3d'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:00.927749 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-558021f8-076f-4b14-afd2-4bf973ae2435 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59b2fdc636d04757a8a942111b6cfe3d'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:00.928353 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-558021f8-076f-4b14-afd2-4bf973ae2435 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59b2fdc636d04757a8a942111b6cfe3d'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:00.940705 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-558021f8-076f-4b14-afd2-4bf973ae2435 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59b2fdc636d04757a8a942111b6cfe3d'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:00.941834 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:55:00.956132 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1257/2502] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:55:00 2022] DELETE /identity/v3/users/59b2fdc636d04757a8a942111b6cfe3d => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:55:01.350498 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec087d5b-9163-42fd-be55-412d694a42ae admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:01.352122 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-ec087d5b-9163-42fd-be55-412d694a42ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:55:01.365635 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec087d5b-9163-42fd-be55-412d694a42ae admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:01.422545 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfde300-c771-4cf7-ac50-cfdf4cbc1dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:01.423174 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfde300-c771-4cf7-ac50-cfdf4cbc1dab admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:01.423477 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfde300-c771-4cf7-ac50-cfdf4cbc1dab admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:01.423751 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfde300-c771-4cf7-ac50-cfdf4cbc1dab admin admin] PATH_INFO: `/v3/projects/9d0bbffeca494c07a340e40ee2a5c42b` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:01.424872 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfde300-c771-4cf7-ac50-cfdf4cbc1dab admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d0bbffeca494c07a340e40ee2a5c42b)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:01.426776 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfde300-c771-4cf7-ac50-cfdf4cbc1dab admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:01.472550 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-6bfde300-c771-4cf7-ac50-cfdf4cbc1dab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d0bbffeca494c07a340e40ee2a5c42b', 'request_id': 'req-6bfde300-c771-4cf7-ac50-cfdf4cbc1dab'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:01.479117 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-6bfde300-c771-4cf7-ac50-cfdf4cbc1dab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d0bbffeca494c07a340e40ee2a5c42b', 'request_id': 'req-6bfde300-c771-4cf7-ac50-cfdf4cbc1dab'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:01.487612 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-6bfde300-c771-4cf7-ac50-cfdf4cbc1dab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d0bbffeca494c07a340e40ee2a5c42b', 'request_id': 'req-6bfde300-c771-4cf7-ac50-cfdf4cbc1dab'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:01.498936 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-6bfde300-c771-4cf7-ac50-cfdf4cbc1dab admin admin] The token cache is being invalidate because project 9d0bbffeca494c07a340e40ee2a5c42b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90653) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:55:01.499219 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-6bfde300-c771-4cf7-ac50-cfdf4cbc1dab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:01.499877 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-6bfde300-c771-4cf7-ac50-cfdf4cbc1dab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:01.501323 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1246/2503] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:55:01 2022] DELETE /identity/v3/projects/9d0bbffeca494c07a340e40ee2a5c42b => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:55:01.506147 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf7e688-d9d8-4d57-a313-1836c798d5cc None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:01.506632 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf7e688-d9d8-4d57-a313-1836c798d5cc None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:01.506732 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf7e688-d9d8-4d57-a313-1836c798d5cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:01.526218 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-4bf7e688-d9d8-4d57-a313-1836c798d5cc None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:55:01.527398 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-4bf7e688-d9d8-4d57-a313-1836c798d5cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:55:01.584961 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1258/2504] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:55:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:55:01.593960 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f297ead-b453-4a44-b92c-e755c2fe668d None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:01.594250 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f297ead-b453-4a44-b92c-e755c2fe668d None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:01.594523 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f297ead-b453-4a44-b92c-e755c2fe668d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:01.614309 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-0f297ead-b453-4a44-b92c-e755c2fe668d None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:55:01.615539 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-0f297ead-b453-4a44-b92c-e755c2fe668d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:55:01.623997 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1247/2505] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:55:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:55:01.628148 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf7e688-d9d8-4d57-a313-1836c798d5cc None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:01.638329 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf7e688-d9d8-4d57-a313-1836c798d5cc None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:01.642933 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7cdd7d-0365-4337-b262-bbd222b79e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:01.643571 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7cdd7d-0365-4337-b262-bbd222b79e25 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:01.643855 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7cdd7d-0365-4337-b262-bbd222b79e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:01.644131 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7cdd7d-0365-4337-b262-bbd222b79e25 admin admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:01.644772 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7cdd7d-0365-4337-b262-bbd222b79e25 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:55:01.645076 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7cdd7d-0365-4337-b262-bbd222b79e25 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:01.646672 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7cdd7d-0365-4337-b262-bbd222b79e25 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:01.658198 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1259/2506] 10.222.0.15 () {58 vars in 1153 bytes} [Sun Sep 11 10:55:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:55:01.662407 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f297ead-b453-4a44-b92c-e755c2fe668d None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:01.671813 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f297ead-b453-4a44-b92c-e755c2fe668d None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:01.676404 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b991db81-3124-477e-b447-5b6ab0704172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:01.676995 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b991db81-3124-477e-b447-5b6ab0704172 admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:01.677299 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b991db81-3124-477e-b447-5b6ab0704172 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:01.677568 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b991db81-3124-477e-b447-5b6ab0704172 admin admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:01.678191 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b991db81-3124-477e-b447-5b6ab0704172 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:01.680113 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b991db81-3124-477e-b447-5b6ab0704172 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:01.707579 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1248/2507] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:55:01 2022] POST /identity/v3/projects => generated 357 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 10:55:01.711615 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7cdd7d-0365-4337-b262-bbd222b79e25 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:01.721519 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7cdd7d-0365-4337-b262-bbd222b79e25 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:01.777306 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa86f58f-94de-4ffb-91ce-83f5e074e18c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:01.777898 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa86f58f-94de-4ffb-91ce-83f5e074e18c admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:01.778177 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa86f58f-94de-4ffb-91ce-83f5e074e18c admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:01.779081 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa86f58f-94de-4ffb-91ce-83f5e074e18c admin admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:01.779796 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa86f58f-94de-4ffb-91ce-83f5e074e18c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:01.781728 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa86f58f-94de-4ffb-91ce-83f5e074e18c admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:01.803279 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1260/2508] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:55:01 2022] POST /identity/v3/users => generated 279 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 10:55:01.808207 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b991db81-3124-477e-b447-5b6ab0704172 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:01.818824 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b991db81-3124-477e-b447-5b6ab0704172 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:01.823736 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6344cf18-61e6-43a6-8f49-d9a67ccd88e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:01.824348 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6344cf18-61e6-43a6-8f49-d9a67ccd88e6 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:01.824629 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6344cf18-61e6-43a6-8f49-d9a67ccd88e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:01.824901 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6344cf18-61e6-43a6-8f49-d9a67ccd88e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:01.825540 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6344cf18-61e6-43a6-8f49-d9a67ccd88e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:01.827131 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6344cf18-61e6-43a6-8f49-d9a67ccd88e6 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:01.837271 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1249/2509] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:55:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 10:55:01.841027 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa86f58f-94de-4ffb-91ce-83f5e074e18c admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:01.850819 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa86f58f-94de-4ffb-91ce-83f5e074e18c admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:01.856017 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff2cc69-8912-4bde-9480-805bad3667fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:01.856613 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff2cc69-8912-4bde-9480-805bad3667fd admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:01.856896 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff2cc69-8912-4bde-9480-805bad3667fd admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:01.857167 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff2cc69-8912-4bde-9480-805bad3667fd admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:01.857846 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff2cc69-8912-4bde-9480-805bad3667fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:01.859871 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff2cc69-8912-4bde-9480-805bad3667fd admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:01.870989 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1261/2510] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:55:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:55:01.875173 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6344cf18-61e6-43a6-8f49-d9a67ccd88e6 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:01.885390 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6344cf18-61e6-43a6-8f49-d9a67ccd88e6 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:01.890017 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6c64b2-7796-4abf-a6a2-17512fe4b5d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:01.890681 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6c64b2-7796-4abf-a6a2-17512fe4b5d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:01.890957 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6c64b2-7796-4abf-a6a2-17512fe4b5d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:01.891224 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6c64b2-7796-4abf-a6a2-17512fe4b5d4 admin admin] PATH_INFO: `/v3/projects/ccfe094152914bb99ff2aee6b3258c09/users/1e5a101d64ca4945807d5f80b197df8a/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:01.906045 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6c64b2-7796-4abf-a6a2-17512fe4b5d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccfe094152914bb99ff2aee6b3258c09, user_id=1e5a101d64ca4945807d5f80b197df8a, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:01.908711 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6c64b2-7796-4abf-a6a2-17512fe4b5d4 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:01.922318 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1250/2511] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:55:01 2022] PUT /identity/v3/projects/ccfe094152914bb99ff2aee6b3258c09/users/1e5a101d64ca4945807d5f80b197df8a/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 10:55:01.933267 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae376eb-985f-4843-a6d4-2e9e0dbfb07b None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:01.933558 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae376eb-985f-4843-a6d4-2e9e0dbfb07b None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:01.933832 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae376eb-985f-4843-a6d4-2e9e0dbfb07b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:01.961208 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-eae376eb-985f-4843-a6d4-2e9e0dbfb07b None None] MFA Rules not processed for user `1e5a101d64ca4945807d5f80b197df8a`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:55:01.962654 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-eae376eb-985f-4843-a6d4-2e9e0dbfb07b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:55:02.007343 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:55:02.028504 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1262/2512] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:55:01 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:55:02.032979 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6c64b2-7796-4abf-a6a2-17512fe4b5d4 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:02.059307 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:55:02.097018 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6c64b2-7796-4abf-a6a2-17512fe4b5d4 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:02.101838 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773695a0-b03f-49a4-8547-28d74abfb052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:02.102486 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773695a0-b03f-49a4-8547-28d74abfb052 admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:02.103521 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773695a0-b03f-49a4-8547-28d74abfb052 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:02.103883 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773695a0-b03f-49a4-8547-28d74abfb052 admin admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:02.104437 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773695a0-b03f-49a4-8547-28d74abfb052 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:02.106381 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773695a0-b03f-49a4-8547-28d74abfb052 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:02.133124 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1251/2513] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:55:02 2022] POST /identity/v3/projects => generated 355 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:55:02.137273 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae376eb-985f-4843-a6d4-2e9e0dbfb07b None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:02.147869 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae376eb-985f-4843-a6d4-2e9e0dbfb07b None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:02.203628 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b15fbb-1662-4937-bf95-b44b332add54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:02.204241 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b15fbb-1662-4937-bf95-b44b332add54 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:02.204527 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b15fbb-1662-4937-bf95-b44b332add54 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:02.204805 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b15fbb-1662-4937-bf95-b44b332add54 admin admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:02.205433 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b15fbb-1662-4937-bf95-b44b332add54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:02.207349 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b15fbb-1662-4937-bf95-b44b332add54 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:02.230446 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1263/2514] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:55:02 2022] POST /identity/v3/users => generated 284 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:55:02.234758 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773695a0-b03f-49a4-8547-28d74abfb052 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:02.245273 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773695a0-b03f-49a4-8547-28d74abfb052 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:02.249939 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96be2dc-c82e-45de-8fc5-b57f2fbea834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:02.250542 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96be2dc-c82e-45de-8fc5-b57f2fbea834 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:02.250817 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96be2dc-c82e-45de-8fc5-b57f2fbea834 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:02.251093 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96be2dc-c82e-45de-8fc5-b57f2fbea834 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:02.251768 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96be2dc-c82e-45de-8fc5-b57f2fbea834 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:02.253392 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96be2dc-c82e-45de-8fc5-b57f2fbea834 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:02.264516 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1252/2515] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:55:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:55:02.268445 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b15fbb-1662-4937-bf95-b44b332add54 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:02.278864 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b15fbb-1662-4937-bf95-b44b332add54 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:02.283523 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c544ca8a-943b-4206-8536-fafa906f54f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:02.284181 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c544ca8a-943b-4206-8536-fafa906f54f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:02.284461 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c544ca8a-943b-4206-8536-fafa906f54f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:02.284735 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c544ca8a-943b-4206-8536-fafa906f54f9 admin admin] PATH_INFO: `/v3/projects/d7d9bc74441d48f487b9fe438311c6e5/users/1de96877abb54ac2ae7125af921a32c1/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:02.300072 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c544ca8a-943b-4206-8536-fafa906f54f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7d9bc74441d48f487b9fe438311c6e5, user_id=1de96877abb54ac2ae7125af921a32c1, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:02.302735 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c544ca8a-943b-4206-8536-fafa906f54f9 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:02.316113 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1264/2516] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:55:02 2022] PUT /identity/v3/projects/d7d9bc74441d48f487b9fe438311c6e5/users/1de96877abb54ac2ae7125af921a32c1/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:55:02.320190 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96be2dc-c82e-45de-8fc5-b57f2fbea834 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:02.330420 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96be2dc-c82e-45de-8fc5-b57f2fbea834 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:02.387243 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbea021f-5290-4b2c-8842-e5f109f44218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:02.387801 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbea021f-5290-4b2c-8842-e5f109f44218 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:02.388085 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbea021f-5290-4b2c-8842-e5f109f44218 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:02.388361 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbea021f-5290-4b2c-8842-e5f109f44218 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:02.388993 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbea021f-5290-4b2c-8842-e5f109f44218 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:02.390624 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbea021f-5290-4b2c-8842-e5f109f44218 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:02.400570 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1253/2517] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:55:02 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:55:02.405075 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c544ca8a-943b-4206-8536-fafa906f54f9 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:02.416044 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c544ca8a-943b-4206-8536-fafa906f54f9 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:02.420951 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7625ad5b-3821-4276-b6da-68107b578f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:02.421608 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7625ad5b-3821-4276-b6da-68107b578f11 admin admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:02.421891 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7625ad5b-3821-4276-b6da-68107b578f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:02.422169 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7625ad5b-3821-4276-b6da-68107b578f11 admin admin] PATH_INFO: `/v3/projects/d7d9bc74441d48f487b9fe438311c6e5/users/1de96877abb54ac2ae7125af921a32c1/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:02.424171 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7625ad5b-3821-4276-b6da-68107b578f11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7d9bc74441d48f487b9fe438311c6e5, user_id=1de96877abb54ac2ae7125af921a32c1, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:02.426829 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7625ad5b-3821-4276-b6da-68107b578f11 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:02.437066 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1265/2518] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:55:02 2022] PUT /identity/v3/projects/d7d9bc74441d48f487b9fe438311c6e5/users/1de96877abb54ac2ae7125af921a32c1/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:55:02.448665 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a87c6-4fac-4fa2-b311-cb423b5ff410 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:02.448955 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a87c6-4fac-4fa2-b311-cb423b5ff410 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:02.449235 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a87c6-4fac-4fa2-b311-cb423b5ff410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:02.477647 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-2c2a87c6-4fac-4fa2-b311-cb423b5ff410 None None] MFA Rules not processed for user `1de96877abb54ac2ae7125af921a32c1`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:55:02.479526 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-2c2a87c6-4fac-4fa2-b311-cb423b5ff410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:55:02.546749 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1254/2519] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:55:02 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:55:02.557942 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7625ad5b-3821-4276-b6da-68107b578f11 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:02.559760 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-7625ad5b-3821-4276-b6da-68107b578f11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:55:02.616325 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7625ad5b-3821-4276-b6da-68107b578f11 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:02.619979 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c30481-f1fc-487b-94f2-75dffb141d47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:02.620578 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c30481-f1fc-487b-94f2-75dffb141d47 service cinder] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:02.620852 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c30481-f1fc-487b-94f2-75dffb141d47 service cinder] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:02.621122 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c30481-f1fc-487b-94f2-75dffb141d47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:02.630056 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c30481-f1fc-487b-94f2-75dffb141d47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:02.631502 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c30481-f1fc-487b-94f2-75dffb141d47 service cinder] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:02.641347 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1266/2520] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:55:02 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:55:02.717728 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2a87c6-4fac-4fa2-b311-cb423b5ff410 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:02.727239 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2a87c6-4fac-4fa2-b311-cb423b5ff410 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:02.731221 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd45a71-1c85-4440-9780-ab70a12ead5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:02.731841 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd45a71-1c85-4440-9780-ab70a12ead5b service cinder] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:02.732123 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd45a71-1c85-4440-9780-ab70a12ead5b service cinder] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:02.732390 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd45a71-1c85-4440-9780-ab70a12ead5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:02.745517 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:55:02.791379 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd45a71-1c85-4440-9780-ab70a12ead5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:02.792841 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd45a71-1c85-4440-9780-ab70a12ead5b service cinder] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:02.801976 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1255/2521] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:55:02 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:55:12.636968 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=77,keystone:INSERT=5 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:55:12.797896 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=20 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:55:13.724741 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e59b181-f269-40b3-ae8f-4429e2b76656 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:13.725219 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e59b181-f269-40b3-ae8f-4429e2b76656 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:13.725328 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e59b181-f269-40b3-ae8f-4429e2b76656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:13.760759 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-5e59b181-f269-40b3-ae8f-4429e2b76656 None None] MFA Rules not processed for user `bb44e7aef0c7436d93a7233307b5f095`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:55:13.762119 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-5e59b181-f269-40b3-ae8f-4429e2b76656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:55:13.839671 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1267/2522] 10.222.0.15 () {58 vars in 967 bytes} [Sun Sep 11 10:55:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:55:13.886433 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd45a71-1c85-4440-9780-ab70a12ead5b service cinder] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:13.887894 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-0cd45a71-1c85-4440-9780-ab70a12ead5b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:55:13.959312 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd45a71-1c85-4440-9780-ab70a12ead5b service cinder] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:13.964897 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0310346c-8ce7-4a2f-ac66-eb151e22d5ca service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7ca566e66a45b29e3d02f47bf84981', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:13.965504 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0310346c-8ce7-4a2f-ac66-eb151e22d5ca service swift] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:13.965784 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0310346c-8ce7-4a2f-ac66-eb151e22d5ca service swift] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:13.966055 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0310346c-8ce7-4a2f-ac66-eb151e22d5ca service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:13.974045 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0310346c-8ce7-4a2f-ac66-eb151e22d5ca service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:13.975666 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0310346c-8ce7-4a2f-ac66-eb151e22d5ca service swift] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:13.983858 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1256/2523] 10.222.0.15 () {60 vars in 1384 bytes} [Sun Sep 11 10:55:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 10:55:17.906165 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e59b181-f269-40b3-ae8f-4429e2b76656 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:17.983130 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e59b181-f269-40b3-ae8f-4429e2b76656 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:17.988592 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a89b808-6155-4c99-9e4d-ba7367b82180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:17.989227 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a89b808-6155-4c99-9e4d-ba7367b82180 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:17.989511 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a89b808-6155-4c99-9e4d-ba7367b82180 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:17.989788 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a89b808-6155-4c99-9e4d-ba7367b82180 admin admin] PATH_INFO: `/v3/users/1e5a101d64ca4945807d5f80b197df8a` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:17.991056 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a89b808-6155-4c99-9e4d-ba7367b82180 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e5a101d64ca4945807d5f80b197df8a)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:17.993217 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a89b808-6155-4c99-9e4d-ba7367b82180 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:18.041786 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-3a89b808-6155-4c99-9e4d-ba7367b82180 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e5a101d64ca4945807d5f80b197df8a'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:18.050058 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-3a89b808-6155-4c99-9e4d-ba7367b82180 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e5a101d64ca4945807d5f80b197df8a'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:18.050673 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-3a89b808-6155-4c99-9e4d-ba7367b82180 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e5a101d64ca4945807d5f80b197df8a'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:18.051278 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-3a89b808-6155-4c99-9e4d-ba7367b82180 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e5a101d64ca4945807d5f80b197df8a'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:18.064441 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-3a89b808-6155-4c99-9e4d-ba7367b82180 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e5a101d64ca4945807d5f80b197df8a'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:18.079643 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1268/2524] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:55:17 2022] DELETE /identity/v3/users/1e5a101d64ca4945807d5f80b197df8a => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:55:18.084759 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe2438-d2f6-4261-ba07-7c6a75dd4542 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:18.085275 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe2438-d2f6-4261-ba07-7c6a75dd4542 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:18.085555 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe2438-d2f6-4261-ba07-7c6a75dd4542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:18.105000 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-84fe2438-d2f6-4261-ba07-7c6a75dd4542 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:55:18.106329 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-84fe2438-d2f6-4261-ba07-7c6a75dd4542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:55:18.165876 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1257/2525] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:55:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 10:55:18.175548 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a89b808-6155-4c99-9e4d-ba7367b82180 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:18.177186 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-3a89b808-6155-4c99-9e4d-ba7367b82180 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:55:18.191306 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:55:18.248680 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a89b808-6155-4c99-9e4d-ba7367b82180 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:18.254756 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8111b832-26f7-41f4-9ed9-5d2051d7cb71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:18.255387 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8111b832-26f7-41f4-9ed9-5d2051d7cb71 service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:18.255701 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8111b832-26f7-41f4-9ed9-5d2051d7cb71 service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:18.255974 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8111b832-26f7-41f4-9ed9-5d2051d7cb71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:18.270609 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8111b832-26f7-41f4-9ed9-5d2051d7cb71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:18.272088 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8111b832-26f7-41f4-9ed9-5d2051d7cb71 service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:18.284186 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1269/2526] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:55:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:55:18.637285 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fe2438-d2f6-4261-ba07-7c6a75dd4542 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:18.638848 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-84fe2438-d2f6-4261-ba07-7c6a75dd4542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:55:18.660972 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fe2438-d2f6-4261-ba07-7c6a75dd4542 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:18.668149 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f5bc4e-161f-4152-9605-7f79ac3b5524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:18.668874 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f5bc4e-161f-4152-9605-7f79ac3b5524 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:18.669387 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f5bc4e-161f-4152-9605-7f79ac3b5524 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:18.669760 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f5bc4e-161f-4152-9605-7f79ac3b5524 admin admin] PATH_INFO: `/v3/projects/ccfe094152914bb99ff2aee6b3258c09` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:18.671057 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f5bc4e-161f-4152-9605-7f79ac3b5524 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccfe094152914bb99ff2aee6b3258c09)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:18.673610 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f5bc4e-161f-4152-9605-7f79ac3b5524 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:18.736598 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-f5f5bc4e-161f-4152-9605-7f79ac3b5524 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccfe094152914bb99ff2aee6b3258c09', 'request_id': 'req-f5f5bc4e-161f-4152-9605-7f79ac3b5524'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:18.748983 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-f5f5bc4e-161f-4152-9605-7f79ac3b5524 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccfe094152914bb99ff2aee6b3258c09', 'request_id': 'req-f5f5bc4e-161f-4152-9605-7f79ac3b5524'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:18.758112 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-f5f5bc4e-161f-4152-9605-7f79ac3b5524 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccfe094152914bb99ff2aee6b3258c09', 'request_id': 'req-f5f5bc4e-161f-4152-9605-7f79ac3b5524'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:18.772681 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-f5f5bc4e-161f-4152-9605-7f79ac3b5524 admin admin] The token cache is being invalidate because project ccfe094152914bb99ff2aee6b3258c09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90653) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:55:18.772975 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-f5f5bc4e-161f-4152-9605-7f79ac3b5524 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:18.773633 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-f5f5bc4e-161f-4152-9605-7f79ac3b5524 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:18.776352 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1258/2527] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:55:18 2022] DELETE /identity/v3/projects/ccfe094152914bb99ff2aee6b3258c09 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:55:18.781249 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8111b832-26f7-41f4-9ed9-5d2051d7cb71 service neutron] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:18.792053 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8111b832-26f7-41f4-9ed9-5d2051d7cb71 service neutron] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:18.860309 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf8c22b-f148-452a-80e3-85ccb0e994dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:18.860955 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf8c22b-f148-452a-80e3-85ccb0e994dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:18.861238 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf8c22b-f148-452a-80e3-85ccb0e994dc admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:18.861512 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf8c22b-f148-452a-80e3-85ccb0e994dc admin admin] PATH_INFO: `/v3/users/1de96877abb54ac2ae7125af921a32c1` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:18.862664 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf8c22b-f148-452a-80e3-85ccb0e994dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=1de96877abb54ac2ae7125af921a32c1)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:18.864658 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf8c22b-f148-452a-80e3-85ccb0e994dc admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:18.920702 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-9cf8c22b-f148-452a-80e3-85ccb0e994dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1de96877abb54ac2ae7125af921a32c1'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:18.931915 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-9cf8c22b-f148-452a-80e3-85ccb0e994dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1de96877abb54ac2ae7125af921a32c1'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:18.932625 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-9cf8c22b-f148-452a-80e3-85ccb0e994dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1de96877abb54ac2ae7125af921a32c1'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:18.933262 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-9cf8c22b-f148-452a-80e3-85ccb0e994dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1de96877abb54ac2ae7125af921a32c1'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:18.955583 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-9cf8c22b-f148-452a-80e3-85ccb0e994dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1de96877abb54ac2ae7125af921a32c1'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:18.976163 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1270/2528] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:55:18 2022] DELETE /identity/v3/users/1de96877abb54ac2ae7125af921a32c1 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:55:19.314001 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f5bc4e-161f-4152-9605-7f79ac3b5524 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:19.316161 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-f5f5bc4e-161f-4152-9605-7f79ac3b5524 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:55:19.342039 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f5bc4e-161f-4152-9605-7f79ac3b5524 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:19.359780 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:55:19.430467 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4b9b78-4856-4054-891f-7da2038008ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:19.431623 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4b9b78-4856-4054-891f-7da2038008ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:19.432125 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4b9b78-4856-4054-891f-7da2038008ff admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:19.432626 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4b9b78-4856-4054-891f-7da2038008ff admin admin] PATH_INFO: `/v3/projects/d7d9bc74441d48f487b9fe438311c6e5` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:19.434849 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4b9b78-4856-4054-891f-7da2038008ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7d9bc74441d48f487b9fe438311c6e5)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:19.436788 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4b9b78-4856-4054-891f-7da2038008ff admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:19.490959 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-6f4b9b78-4856-4054-891f-7da2038008ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7d9bc74441d48f487b9fe438311c6e5', 'request_id': 'req-6f4b9b78-4856-4054-891f-7da2038008ff'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:19.498350 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-6f4b9b78-4856-4054-891f-7da2038008ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7d9bc74441d48f487b9fe438311c6e5', 'request_id': 'req-6f4b9b78-4856-4054-891f-7da2038008ff'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:19.507698 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-6f4b9b78-4856-4054-891f-7da2038008ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7d9bc74441d48f487b9fe438311c6e5', 'request_id': 'req-6f4b9b78-4856-4054-891f-7da2038008ff'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:19.520758 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-6f4b9b78-4856-4054-891f-7da2038008ff admin admin] The token cache is being invalidate because project d7d9bc74441d48f487b9fe438311c6e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90653) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:55:19.521052 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-6f4b9b78-4856-4054-891f-7da2038008ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:19.522232 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-6f4b9b78-4856-4054-891f-7da2038008ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:19.524135 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1259/2529] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:55:19 2022] DELETE /identity/v3/projects/d7d9bc74441d48f487b9fe438311c6e5 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:55:19.529341 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f3e35e-36e2-40fd-a3cd-106beb2a8154 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:19.530263 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f3e35e-36e2-40fd-a3cd-106beb2a8154 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:19.530263 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f3e35e-36e2-40fd-a3cd-106beb2a8154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:19.552326 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-c2f3e35e-36e2-40fd-a3cd-106beb2a8154 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:55:19.553828 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-c2f3e35e-36e2-40fd-a3cd-106beb2a8154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:55:19.583094 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:55:19.632169 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1271/2530] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:55:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:55:19.640514 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903b02ea-c8a1-4d24-8635-ecdf1fa81ba9 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:19.640802 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903b02ea-c8a1-4d24-8635-ecdf1fa81ba9 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:19.641076 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903b02ea-c8a1-4d24-8635-ecdf1fa81ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:19.671071 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-903b02ea-c8a1-4d24-8635-ecdf1fa81ba9 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:55:19.672300 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-903b02ea-c8a1-4d24-8635-ecdf1fa81ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:55:19.681210 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1260/2531] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:55:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 10:55:19.685572 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f3e35e-36e2-40fd-a3cd-106beb2a8154 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:19.695390 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f3e35e-36e2-40fd-a3cd-106beb2a8154 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:19.701173 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb42e9d-3743-4a07-988d-e595b1fa83f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:19.701787 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb42e9d-3743-4a07-988d-e595b1fa83f1 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:19.702132 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb42e9d-3743-4a07-988d-e595b1fa83f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:19.702468 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb42e9d-3743-4a07-988d-e595b1fa83f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:19.703174 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb42e9d-3743-4a07-988d-e595b1fa83f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:55:19.703576 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb42e9d-3743-4a07-988d-e595b1fa83f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:19.705239 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb42e9d-3743-4a07-988d-e595b1fa83f1 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:19.718474 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1272/2532] 10.222.0.15 () {58 vars in 1153 bytes} [Sun Sep 11 10:55:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:55:19.722856 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903b02ea-c8a1-4d24-8635-ecdf1fa81ba9 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:19.732975 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903b02ea-c8a1-4d24-8635-ecdf1fa81ba9 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:19.738062 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc033e8-260b-4e2b-8fb4-9926a6ba0f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:19.738673 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc033e8-260b-4e2b-8fb4-9926a6ba0f10 admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:19.738959 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc033e8-260b-4e2b-8fb4-9926a6ba0f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:19.739287 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc033e8-260b-4e2b-8fb4-9926a6ba0f10 admin admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:19.739999 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc033e8-260b-4e2b-8fb4-9926a6ba0f10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:19.742000 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc033e8-260b-4e2b-8fb4-9926a6ba0f10 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:19.773353 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1261/2533] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:55:19 2022] POST /identity/v3/projects => generated 387 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 10:55:19.774489 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb42e9d-3743-4a07-988d-e595b1fa83f1 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:19.785920 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb42e9d-3743-4a07-988d-e595b1fa83f1 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:19.844071 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3c3732-672b-4874-89f7-6f588abb7aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:19.844684 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3c3732-672b-4874-89f7-6f588abb7aaa admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:19.844963 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3c3732-672b-4874-89f7-6f588abb7aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:19.845236 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3c3732-672b-4874-89f7-6f588abb7aaa admin admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:19.845868 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3c3732-672b-4874-89f7-6f588abb7aaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:19.847816 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3c3732-672b-4874-89f7-6f588abb7aaa admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:19.874260 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1273/2534] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:55:19 2022] POST /identity/v3/users => generated 294 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 10:55:19.875898 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc033e8-260b-4e2b-8fb4-9926a6ba0f10 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:19.888054 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc033e8-260b-4e2b-8fb4-9926a6ba0f10 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:19.892735 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8fa2a6-684a-4fdd-b487-2e424328a25b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:19.893350 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8fa2a6-684a-4fdd-b487-2e424328a25b admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:19.893636 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8fa2a6-684a-4fdd-b487-2e424328a25b admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:19.893913 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8fa2a6-684a-4fdd-b487-2e424328a25b admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:19.894564 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8fa2a6-684a-4fdd-b487-2e424328a25b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:19.896213 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8fa2a6-684a-4fdd-b487-2e424328a25b admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:19.911687 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1262/2535] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:55:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 10:55:19.912823 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3c3732-672b-4874-89f7-6f588abb7aaa admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:19.924364 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3c3732-672b-4874-89f7-6f588abb7aaa admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:19.930872 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf2c565-6ed5-414a-aab9-28fb6de8a1b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:19.931504 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf2c565-6ed5-414a-aab9-28fb6de8a1b7 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:19.931831 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf2c565-6ed5-414a-aab9-28fb6de8a1b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:19.932060 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf2c565-6ed5-414a-aab9-28fb6de8a1b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:19.932860 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf2c565-6ed5-414a-aab9-28fb6de8a1b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:19.934300 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf2c565-6ed5-414a-aab9-28fb6de8a1b7 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:19.950910 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8fa2a6-684a-4fdd-b487-2e424328a25b admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:19.956353 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1274/2536] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:55:19 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:55:19.964366 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8fa2a6-684a-4fdd-b487-2e424328a25b admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:19.969393 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f389309-e967-48e6-8700-9c717012fe6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:19.970192 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f389309-e967-48e6-8700-9c717012fe6d admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:19.970575 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f389309-e967-48e6-8700-9c717012fe6d admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:19.970944 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f389309-e967-48e6-8700-9c717012fe6d admin admin] PATH_INFO: `/v3/projects/d0d6477d0a7946109e05f1368e5006d3/users/6e4caaa2000c4b13b78487118ec4872c/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:20.019958 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f389309-e967-48e6-8700-9c717012fe6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0d6477d0a7946109e05f1368e5006d3, user_id=6e4caaa2000c4b13b78487118ec4872c, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:20.023671 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f389309-e967-48e6-8700-9c717012fe6d admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:20.064556 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d714f5de-dcd1-4ac3-ab2d-010fad68843c None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:20.064842 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d714f5de-dcd1-4ac3-ab2d-010fad68843c None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:20.065118 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d714f5de-dcd1-4ac3-ab2d-010fad68843c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:20.067508 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1263/2537] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:55:19 2022] PUT /identity/v3/projects/d0d6477d0a7946109e05f1368e5006d3/users/6e4caaa2000c4b13b78487118ec4872c/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:55:20.133851 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-d714f5de-dcd1-4ac3-ab2d-010fad68843c None None] MFA Rules not processed for user `6e4caaa2000c4b13b78487118ec4872c`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:55:20.135575 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-d714f5de-dcd1-4ac3-ab2d-010fad68843c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:55:20.249804 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1275/2538] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:55:20 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:55:20.254054 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f389309-e967-48e6-8700-9c717012fe6d admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:20.315127 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:55:20.340924 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f389309-e967-48e6-8700-9c717012fe6d admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:20.345767 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec837b36-e7d7-4c2b-9b42-d26bcbe05d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:20.346378 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec837b36-e7d7-4c2b-9b42-d26bcbe05d2b admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:20.346656 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec837b36-e7d7-4c2b-9b42-d26bcbe05d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:20.346923 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec837b36-e7d7-4c2b-9b42-d26bcbe05d2b admin admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:20.347590 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec837b36-e7d7-4c2b-9b42-d26bcbe05d2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:20.349506 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec837b36-e7d7-4c2b-9b42-d26bcbe05d2b admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:20.395460 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1264/2539] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:55:20 2022] POST /identity/v3/projects => generated 383 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:55:20.396746 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d714f5de-dcd1-4ac3-ab2d-010fad68843c None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:20.408998 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d714f5de-dcd1-4ac3-ab2d-010fad68843c None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:20.441706 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:55:20.503874 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fe7099-8a30-4343-9a48-3ae080d66f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:20.504614 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe7099-8a30-4343-9a48-3ae080d66f4f admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:20.504967 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe7099-8a30-4343-9a48-3ae080d66f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:20.505309 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe7099-8a30-4343-9a48-3ae080d66f4f admin admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:20.506026 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fe7099-8a30-4343-9a48-3ae080d66f4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:20.518716 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fe7099-8a30-4343-9a48-3ae080d66f4f admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:20.550066 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1276/2540] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:55:20 2022] POST /identity/v3/users => generated 298 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:55:20.553963 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec837b36-e7d7-4c2b-9b42-d26bcbe05d2b admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:20.571848 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec837b36-e7d7-4c2b-9b42-d26bcbe05d2b admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:20.581427 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb0bb02-8898-4931-b96a-9d8fdd3ed64c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:20.582261 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb0bb02-8898-4931-b96a-9d8fdd3ed64c admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:20.582701 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb0bb02-8898-4931-b96a-9d8fdd3ed64c admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:20.582982 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb0bb02-8898-4931-b96a-9d8fdd3ed64c admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:20.583848 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb0bb02-8898-4931-b96a-9d8fdd3ed64c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:20.585931 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb0bb02-8898-4931-b96a-9d8fdd3ed64c admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:20.601337 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1265/2541] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:55:20 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:55:20.604694 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fe7099-8a30-4343-9a48-3ae080d66f4f admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:20.617950 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fe7099-8a30-4343-9a48-3ae080d66f4f admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:20.626695 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a31fd52-6099-407b-b408-6cc7eacd654e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:20.628006 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a31fd52-6099-407b-b408-6cc7eacd654e admin admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:20.628395 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a31fd52-6099-407b-b408-6cc7eacd654e admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:20.628775 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a31fd52-6099-407b-b408-6cc7eacd654e admin admin] PATH_INFO: `/v3/projects/a809c6b29abd4b04b90068e6e5d92c8f/users/6f03fb72192d45f397ded92b6a8a95c8/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:20.653210 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a31fd52-6099-407b-b408-6cc7eacd654e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a809c6b29abd4b04b90068e6e5d92c8f, user_id=6f03fb72192d45f397ded92b6a8a95c8, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:20.656327 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a31fd52-6099-407b-b408-6cc7eacd654e admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:20.679334 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1277/2542] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:55:20 2022] PUT /identity/v3/projects/a809c6b29abd4b04b90068e6e5d92c8f/users/6f03fb72192d45f397ded92b6a8a95c8/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 10:55:20.684106 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb0bb02-8898-4931-b96a-9d8fdd3ed64c admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:20.694530 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb0bb02-8898-4931-b96a-9d8fdd3ed64c admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:20.777152 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993d2430-344b-4a1a-88eb-eee5af538be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:20.778932 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993d2430-344b-4a1a-88eb-eee5af538be6 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:20.779375 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993d2430-344b-4a1a-88eb-eee5af538be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:20.779922 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993d2430-344b-4a1a-88eb-eee5af538be6 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:20.792360 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993d2430-344b-4a1a-88eb-eee5af538be6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:20.794009 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993d2430-344b-4a1a-88eb-eee5af538be6 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:20.807497 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1266/2543] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:55:20 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:55:20.812465 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a31fd52-6099-407b-b408-6cc7eacd654e admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:20.836531 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a31fd52-6099-407b-b408-6cc7eacd654e admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:20.841588 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837b7fe7-3286-4209-9f08-4695eb2eb848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:20.842358 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837b7fe7-3286-4209-9f08-4695eb2eb848 admin admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:20.842707 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837b7fe7-3286-4209-9f08-4695eb2eb848 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:20.842980 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837b7fe7-3286-4209-9f08-4695eb2eb848 admin admin] PATH_INFO: `/v3/projects/a809c6b29abd4b04b90068e6e5d92c8f/users/6f03fb72192d45f397ded92b6a8a95c8/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:20.844983 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837b7fe7-3286-4209-9f08-4695eb2eb848 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a809c6b29abd4b04b90068e6e5d92c8f, user_id=6f03fb72192d45f397ded92b6a8a95c8, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:20.847681 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837b7fe7-3286-4209-9f08-4695eb2eb848 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:20.860138 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1278/2544] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:55:20 2022] PUT /identity/v3/projects/a809c6b29abd4b04b90068e6e5d92c8f/users/6f03fb72192d45f397ded92b6a8a95c8/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:55:20.872126 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47478db0-002f-4429-a641-f8073d9657db None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:20.872512 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47478db0-002f-4429-a641-f8073d9657db None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:20.872858 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47478db0-002f-4429-a641-f8073d9657db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:20.909153 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-47478db0-002f-4429-a641-f8073d9657db None None] MFA Rules not processed for user `6f03fb72192d45f397ded92b6a8a95c8`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:55:20.912844 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-47478db0-002f-4429-a641-f8073d9657db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:55:21.011979 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1267/2545] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:55:20 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:55:21.023287 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837b7fe7-3286-4209-9f08-4695eb2eb848 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:21.025157 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-837b7fe7-3286-4209-9f08-4695eb2eb848 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:55:21.114509 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837b7fe7-3286-4209-9f08-4695eb2eb848 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:21.119261 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58959d7a-3160-454e-bdc4-8a990cabf12c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:21.120137 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58959d7a-3160-454e-bdc4-8a990cabf12c service cinder] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:21.120666 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58959d7a-3160-454e-bdc4-8a990cabf12c service cinder] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:21.121038 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58959d7a-3160-454e-bdc4-8a990cabf12c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:21.132053 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58959d7a-3160-454e-bdc4-8a990cabf12c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:21.133584 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47478db0-002f-4429-a641-f8073d9657db None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:21.133719 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58959d7a-3160-454e-bdc4-8a990cabf12c service cinder] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:21.134910 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-47478db0-002f-4429-a641-f8073d9657db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:55:21.145668 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1279/2546] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:55:21 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:55:21.180031 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:55:21.210325 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47478db0-002f-4429-a641-f8073d9657db None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:21.216425 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58959d7a-3160-454e-bdc4-8a990cabf12c service cinder] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:21.216681 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b858196d-9122-470d-8b42-eac8bf0fd69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:21.217362 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b858196d-9122-470d-8b42-eac8bf0fd69e admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:21.217648 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b858196d-9122-470d-8b42-eac8bf0fd69e admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:21.217920 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b858196d-9122-470d-8b42-eac8bf0fd69e admin admin] PATH_INFO: `/v3/users/8902fe9b4ce54ad398422722101a947a` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:21.219050 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b858196d-9122-470d-8b42-eac8bf0fd69e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8902fe9b4ce54ad398422722101a947a)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:21.221011 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b858196d-9122-470d-8b42-eac8bf0fd69e admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:21.228505 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58959d7a-3160-454e-bdc4-8a990cabf12c service cinder] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:21.233632 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ccb086-fb75-4652-8f80-5b53f45b34a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:21.234560 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ccb086-fb75-4652-8f80-5b53f45b34a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:21.234841 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ccb086-fb75-4652-8f80-5b53f45b34a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:21.235105 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ccb086-fb75-4652-8f80-5b53f45b34a3 admin admin] PATH_INFO: `/v3/users/6e4caaa2000c4b13b78487118ec4872c` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:21.236263 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ccb086-fb75-4652-8f80-5b53f45b34a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e4caaa2000c4b13b78487118ec4872c)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:21.238191 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ccb086-fb75-4652-8f80-5b53f45b34a3 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:21.272655 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-b858196d-9122-470d-8b42-eac8bf0fd69e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8902fe9b4ce54ad398422722101a947a'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:21.281502 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-b858196d-9122-470d-8b42-eac8bf0fd69e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8902fe9b4ce54ad398422722101a947a'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:21.282801 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-b858196d-9122-470d-8b42-eac8bf0fd69e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8902fe9b4ce54ad398422722101a947a'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:21.283461 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-b858196d-9122-470d-8b42-eac8bf0fd69e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8902fe9b4ce54ad398422722101a947a'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:21.289767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-79ccb086-fb75-4652-8f80-5b53f45b34a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e4caaa2000c4b13b78487118ec4872c'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:21.291862 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-b858196d-9122-470d-8b42-eac8bf0fd69e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8902fe9b4ce54ad398422722101a947a'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:21.298172 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-79ccb086-fb75-4652-8f80-5b53f45b34a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e4caaa2000c4b13b78487118ec4872c'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:21.299450 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-79ccb086-fb75-4652-8f80-5b53f45b34a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e4caaa2000c4b13b78487118ec4872c'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:21.300127 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-79ccb086-fb75-4652-8f80-5b53f45b34a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e4caaa2000c4b13b78487118ec4872c'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:21.302203 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=86,keystone:INSERT=6,keystone:DELETE=8 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:55:21.310458 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1268/2547] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:55:21 2022] DELETE /identity/v3/users/8902fe9b4ce54ad398422722101a947a => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 10:55:21.317657 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-79ccb086-fb75-4652-8f80-5b53f45b34a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e4caaa2000c4b13b78487118ec4872c'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:21.331014 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1280/2548] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:55:21 2022] DELETE /identity/v3/users/6e4caaa2000c4b13b78487118ec4872c => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 10:55:21.336377 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2373d54-808c-47fe-bf6e-d2a3339f4f7a None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:21.336859 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2373d54-808c-47fe-bf6e-d2a3339f4f7a None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:21.337036 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2373d54-808c-47fe-bf6e-d2a3339f4f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:21.362111 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-c2373d54-808c-47fe-bf6e-d2a3339f4f7a None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:55:21.363612 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-c2373d54-808c-47fe-bf6e-d2a3339f4f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:55:21.427492 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1269/2549] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:55:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:55:21.437918 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ccb086-fb75-4652-8f80-5b53f45b34a3 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:21.439506 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-79ccb086-fb75-4652-8f80-5b53f45b34a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:55:21.491021 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2373d54-808c-47fe-bf6e-d2a3339f4f7a None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:21.492936 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-c2373d54-808c-47fe-bf6e-d2a3339f4f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:55:21.506333 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ccb086-fb75-4652-8f80-5b53f45b34a3 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:21.512100 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9ccf58-f11e-45e5-879d-5fa936bd9db3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:21.512742 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9ccf58-f11e-45e5-879d-5fa936bd9db3 service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:21.513030 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9ccf58-f11e-45e5-879d-5fa936bd9db3 service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:21.513299 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9ccf58-f11e-45e5-879d-5fa936bd9db3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:21.520372 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2373d54-808c-47fe-bf6e-d2a3339f4f7a None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:21.526627 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff48271-ce24-4670-b111-ce4cbd56e0e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:21.527110 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff48271-ce24-4670-b111-ce4cbd56e0e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:21.527394 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff48271-ce24-4670-b111-ce4cbd56e0e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:21.527709 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff48271-ce24-4670-b111-ce4cbd56e0e5 admin admin] PATH_INFO: `/v3/projects/73cbd4a12d444b0ba5d4c37d12f2ea49` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:21.528958 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff48271-ce24-4670-b111-ce4cbd56e0e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73cbd4a12d444b0ba5d4c37d12f2ea49)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:21.530946 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff48271-ce24-4670-b111-ce4cbd56e0e5 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:21.531963 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9ccf58-f11e-45e5-879d-5fa936bd9db3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:21.534099 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9ccf58-f11e-45e5-879d-5fa936bd9db3 service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:21.548235 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1281/2550] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:55:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 10:55:21.585998 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-8ff48271-ce24-4670-b111-ce4cbd56e0e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73cbd4a12d444b0ba5d4c37d12f2ea49', 'request_id': 'req-8ff48271-ce24-4670-b111-ce4cbd56e0e5'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:21.587726 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:55:21.596061 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-8ff48271-ce24-4670-b111-ce4cbd56e0e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73cbd4a12d444b0ba5d4c37d12f2ea49', 'request_id': 'req-8ff48271-ce24-4670-b111-ce4cbd56e0e5'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:21.608576 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-8ff48271-ce24-4670-b111-ce4cbd56e0e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73cbd4a12d444b0ba5d4c37d12f2ea49', 'request_id': 'req-8ff48271-ce24-4670-b111-ce4cbd56e0e5'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:21.624014 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-8ff48271-ce24-4670-b111-ce4cbd56e0e5 admin admin] The token cache is being invalidate because project 73cbd4a12d444b0ba5d4c37d12f2ea49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90653) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:55:21.624286 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-8ff48271-ce24-4670-b111-ce4cbd56e0e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:21.624961 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-8ff48271-ce24-4670-b111-ce4cbd56e0e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:21.626714 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1270/2551] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:55:21 2022] DELETE /identity/v3/projects/73cbd4a12d444b0ba5d4c37d12f2ea49 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:55:21.630484 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9ccf58-f11e-45e5-879d-5fa936bd9db3 service neutron] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:21.640936 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9ccf58-f11e-45e5-879d-5fa936bd9db3 service neutron] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:21.707179 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98fcd55-1c7b-4fb0-99ea-47c5f7d496ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:21.708256 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98fcd55-1c7b-4fb0-99ea-47c5f7d496ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:21.708384 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98fcd55-1c7b-4fb0-99ea-47c5f7d496ba admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:21.708700 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98fcd55-1c7b-4fb0-99ea-47c5f7d496ba admin admin] PATH_INFO: `/v3/users/55afe027d2b046e094eef686ccbbf5e4` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:21.710038 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98fcd55-1c7b-4fb0-99ea-47c5f7d496ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=55afe027d2b046e094eef686ccbbf5e4)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:21.712893 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98fcd55-1c7b-4fb0-99ea-47c5f7d496ba admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:21.767315 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-f98fcd55-1c7b-4fb0-99ea-47c5f7d496ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55afe027d2b046e094eef686ccbbf5e4'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:21.776514 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-f98fcd55-1c7b-4fb0-99ea-47c5f7d496ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55afe027d2b046e094eef686ccbbf5e4'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:21.777104 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-f98fcd55-1c7b-4fb0-99ea-47c5f7d496ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55afe027d2b046e094eef686ccbbf5e4'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:21.777722 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-f98fcd55-1c7b-4fb0-99ea-47c5f7d496ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55afe027d2b046e094eef686ccbbf5e4'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:21.790998 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-f98fcd55-1c7b-4fb0-99ea-47c5f7d496ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55afe027d2b046e094eef686ccbbf5e4'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:21.796566 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:55:21.807911 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1282/2552] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:55:21 2022] DELETE /identity/v3/users/55afe027d2b046e094eef686ccbbf5e4 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:55:21.858473 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff48271-ce24-4670-b111-ce4cbd56e0e5 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:21.860485 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-8ff48271-ce24-4670-b111-ce4cbd56e0e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:55:21.874265 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff48271-ce24-4670-b111-ce4cbd56e0e5 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:21.961260 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d0a7ad-6ea5-4979-b68e-82184de92bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:21.962163 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d0a7ad-6ea5-4979-b68e-82184de92bf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:21.962519 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d0a7ad-6ea5-4979-b68e-82184de92bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:21.962869 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d0a7ad-6ea5-4979-b68e-82184de92bf7 admin admin] PATH_INFO: `/v3/projects/d0d6477d0a7946109e05f1368e5006d3` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:21.964797 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d0a7ad-6ea5-4979-b68e-82184de92bf7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0d6477d0a7946109e05f1368e5006d3)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:21.966976 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d0a7ad-6ea5-4979-b68e-82184de92bf7 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:22.036145 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-17d0a7ad-6ea5-4979-b68e-82184de92bf7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0d6477d0a7946109e05f1368e5006d3', 'request_id': 'req-17d0a7ad-6ea5-4979-b68e-82184de92bf7'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:22.055277 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98fcd55-1c7b-4fb0-99ea-47c5f7d496ba admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:22.057903 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-f98fcd55-1c7b-4fb0-99ea-47c5f7d496ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:55:22.060909 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-17d0a7ad-6ea5-4979-b68e-82184de92bf7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0d6477d0a7946109e05f1368e5006d3', 'request_id': 'req-17d0a7ad-6ea5-4979-b68e-82184de92bf7'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:22.073719 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98fcd55-1c7b-4fb0-99ea-47c5f7d496ba admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:22.077182 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-17d0a7ad-6ea5-4979-b68e-82184de92bf7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0d6477d0a7946109e05f1368e5006d3', 'request_id': 'req-17d0a7ad-6ea5-4979-b68e-82184de92bf7'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:22.095339 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-17d0a7ad-6ea5-4979-b68e-82184de92bf7 admin admin] The token cache is being invalidate because project d0d6477d0a7946109e05f1368e5006d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90653) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:55:22.098345 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-17d0a7ad-6ea5-4979-b68e-82184de92bf7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:22.100117 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-17d0a7ad-6ea5-4979-b68e-82184de92bf7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:22.106898 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1271/2553] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:55:21 2022] DELETE /identity/v3/projects/d0d6477d0a7946109e05f1368e5006d3 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:55:22.110727 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d0a7ad-6ea5-4979-b68e-82184de92bf7 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:22.112194 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-17d0a7ad-6ea5-4979-b68e-82184de92bf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:55:22.148342 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b986d5-f38e-48bc-ab4e-c5838997c551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:22.149488 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b986d5-f38e-48bc-ab4e-c5838997c551 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:22.149488 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b986d5-f38e-48bc-ab4e-c5838997c551 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:22.149836 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b986d5-f38e-48bc-ab4e-c5838997c551 admin admin] PATH_INFO: `/v3/projects/47c13d1196e0434cbb5e0b1469625e87` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:22.151306 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b986d5-f38e-48bc-ab4e-c5838997c551 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47c13d1196e0434cbb5e0b1469625e87)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:22.154069 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b986d5-f38e-48bc-ab4e-c5838997c551 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:22.191580 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d0a7ad-6ea5-4979-b68e-82184de92bf7 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:22.196483 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6069c2d9-a9c1-421f-ac2f-5dcc73d337a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:22.197238 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6069c2d9-a9c1-421f-ac2f-5dcc73d337a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:22.197552 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6069c2d9-a9c1-421f-ac2f-5dcc73d337a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:22.197915 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6069c2d9-a9c1-421f-ac2f-5dcc73d337a7 admin admin] PATH_INFO: `/v3/users/6f03fb72192d45f397ded92b6a8a95c8` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:22.199277 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6069c2d9-a9c1-421f-ac2f-5dcc73d337a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f03fb72192d45f397ded92b6a8a95c8)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:22.201316 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6069c2d9-a9c1-421f-ac2f-5dcc73d337a7 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:22.212258 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-12b986d5-f38e-48bc-ab4e-c5838997c551 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47c13d1196e0434cbb5e0b1469625e87', 'request_id': 'req-12b986d5-f38e-48bc-ab4e-c5838997c551'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:22.215361 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:55:22.222537 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-12b986d5-f38e-48bc-ab4e-c5838997c551 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47c13d1196e0434cbb5e0b1469625e87', 'request_id': 'req-12b986d5-f38e-48bc-ab4e-c5838997c551'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:22.229853 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-12b986d5-f38e-48bc-ab4e-c5838997c551 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47c13d1196e0434cbb5e0b1469625e87', 'request_id': 'req-12b986d5-f38e-48bc-ab4e-c5838997c551'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:22.247053 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-12b986d5-f38e-48bc-ab4e-c5838997c551 admin admin] The token cache is being invalidate because project 47c13d1196e0434cbb5e0b1469625e87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:55:22.247558 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-12b986d5-f38e-48bc-ab4e-c5838997c551 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:22.248316 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-12b986d5-f38e-48bc-ab4e-c5838997c551 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:22.249919 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1283/2554] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:55:22 2022] DELETE /identity/v3/projects/47c13d1196e0434cbb5e0b1469625e87 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 10:55:22.276737 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-6069c2d9-a9c1-421f-ac2f-5dcc73d337a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f03fb72192d45f397ded92b6a8a95c8'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:22.288309 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-6069c2d9-a9c1-421f-ac2f-5dcc73d337a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f03fb72192d45f397ded92b6a8a95c8'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:22.289029 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-6069c2d9-a9c1-421f-ac2f-5dcc73d337a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f03fb72192d45f397ded92b6a8a95c8'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:22.289947 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-6069c2d9-a9c1-421f-ac2f-5dcc73d337a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f03fb72192d45f397ded92b6a8a95c8'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:22.300978 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-6069c2d9-a9c1-421f-ac2f-5dcc73d337a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f03fb72192d45f397ded92b6a8a95c8'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:22.329574 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1272/2555] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:55:22 2022] DELETE /identity/v3/users/6f03fb72192d45f397ded92b6a8a95c8 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 10:55:22.639072 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b986d5-f38e-48bc-ab4e-c5838997c551 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:22.641053 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-12b986d5-f38e-48bc-ab4e-c5838997c551 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:55:22.714975 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b986d5-f38e-48bc-ab4e-c5838997c551 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:22.748075 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:55:22.779701 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64948e45-5cdd-431c-b94e-0b6f34d5f443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:22.780343 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64948e45-5cdd-431c-b94e-0b6f34d5f443 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:22.780623 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64948e45-5cdd-431c-b94e-0b6f34d5f443 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:22.780898 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64948e45-5cdd-431c-b94e-0b6f34d5f443 admin admin] PATH_INFO: `/v3/projects/a809c6b29abd4b04b90068e6e5d92c8f` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:22.782015 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64948e45-5cdd-431c-b94e-0b6f34d5f443 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a809c6b29abd4b04b90068e6e5d92c8f)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:22.783981 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64948e45-5cdd-431c-b94e-0b6f34d5f443 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:22.835536 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-64948e45-5cdd-431c-b94e-0b6f34d5f443 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a809c6b29abd4b04b90068e6e5d92c8f', 'request_id': 'req-64948e45-5cdd-431c-b94e-0b6f34d5f443'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:22.843157 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-64948e45-5cdd-431c-b94e-0b6f34d5f443 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a809c6b29abd4b04b90068e6e5d92c8f', 'request_id': 'req-64948e45-5cdd-431c-b94e-0b6f34d5f443'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:22.849016 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-64948e45-5cdd-431c-b94e-0b6f34d5f443 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a809c6b29abd4b04b90068e6e5d92c8f', 'request_id': 'req-64948e45-5cdd-431c-b94e-0b6f34d5f443'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:22.861963 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-64948e45-5cdd-431c-b94e-0b6f34d5f443 admin admin] The token cache is being invalidate because project a809c6b29abd4b04b90068e6e5d92c8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:55:22.862352 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-64948e45-5cdd-431c-b94e-0b6f34d5f443 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:22.863184 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-64948e45-5cdd-431c-b94e-0b6f34d5f443 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:22.864828 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1284/2556] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:55:22 2022] DELETE /identity/v3/projects/a809c6b29abd4b04b90068e6e5d92c8f => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:55:22.870628 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd7f124-fd49-4a4e-9d20-a59e763f63c5 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:22.871118 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd7f124-fd49-4a4e-9d20-a59e763f63c5 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:22.871302 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd7f124-fd49-4a4e-9d20-a59e763f63c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:22.893208 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-9fd7f124-fd49-4a4e-9d20-a59e763f63c5 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:55:22.894777 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-9fd7f124-fd49-4a4e-9d20-a59e763f63c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:55:22.962615 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1273/2557] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:55:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:55:22.970278 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895e298-1c77-4410-adb8-5a9b2825ddca None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:22.970563 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895e298-1c77-4410-adb8-5a9b2825ddca None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:22.970838 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895e298-1c77-4410-adb8-5a9b2825ddca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:22.991043 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-f895e298-1c77-4410-adb8-5a9b2825ddca None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:55:22.992274 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-f895e298-1c77-4410-adb8-5a9b2825ddca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:55:23.001671 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1285/2558] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:55:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:55:23.005913 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd7f124-fd49-4a4e-9d20-a59e763f63c5 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:23.016170 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd7f124-fd49-4a4e-9d20-a59e763f63c5 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:23.020708 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323dc0bb-b6a8-42e2-85f6-1c9bbce09e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:23.021302 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323dc0bb-b6a8-42e2-85f6-1c9bbce09e01 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:23.021575 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323dc0bb-b6a8-42e2-85f6-1c9bbce09e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:23.021846 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323dc0bb-b6a8-42e2-85f6-1c9bbce09e01 admin admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:23.022481 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323dc0bb-b6a8-42e2-85f6-1c9bbce09e01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:55:23.022776 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323dc0bb-b6a8-42e2-85f6-1c9bbce09e01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:23.024440 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323dc0bb-b6a8-42e2-85f6-1c9bbce09e01 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:23.036364 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1274/2559] 10.222.0.15 () {58 vars in 1153 bytes} [Sun Sep 11 10:55:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:55:23.040653 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f895e298-1c77-4410-adb8-5a9b2825ddca None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:23.049803 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f895e298-1c77-4410-adb8-5a9b2825ddca None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:23.054362 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062d83df-95d2-416a-ad19-6a5178606486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:23.054956 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062d83df-95d2-416a-ad19-6a5178606486 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:23.055234 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062d83df-95d2-416a-ad19-6a5178606486 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:23.055539 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062d83df-95d2-416a-ad19-6a5178606486 admin admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:23.056299 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062d83df-95d2-416a-ad19-6a5178606486 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:23.058291 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062d83df-95d2-416a-ad19-6a5178606486 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:23.085181 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1286/2560] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:55:23 2022] POST /identity/v3/projects => generated 387 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:55:23.096809 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323dc0bb-b6a8-42e2-85f6-1c9bbce09e01 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:23.108182 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323dc0bb-b6a8-42e2-85f6-1c9bbce09e01 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:23.168910 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d7ed5d-fb7d-469e-9dcb-e7c0b31af978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:23.169523 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d7ed5d-fb7d-469e-9dcb-e7c0b31af978 admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:23.169799 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d7ed5d-fb7d-469e-9dcb-e7c0b31af978 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:23.170069 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d7ed5d-fb7d-469e-9dcb-e7c0b31af978 admin admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:23.171134 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d7ed5d-fb7d-469e-9dcb-e7c0b31af978 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:23.172658 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d7ed5d-fb7d-469e-9dcb-e7c0b31af978 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:23.196405 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1275/2561] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:55:23 2022] POST /identity/v3/users => generated 294 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 10:55:23.200985 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062d83df-95d2-416a-ad19-6a5178606486 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:23.211580 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062d83df-95d2-416a-ad19-6a5178606486 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:23.216569 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264339cd-8b16-42f7-b46c-e8889e8c04fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:23.217181 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264339cd-8b16-42f7-b46c-e8889e8c04fe admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:23.217461 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264339cd-8b16-42f7-b46c-e8889e8c04fe admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:23.217732 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264339cd-8b16-42f7-b46c-e8889e8c04fe admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:23.218419 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264339cd-8b16-42f7-b46c-e8889e8c04fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:23.220057 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264339cd-8b16-42f7-b46c-e8889e8c04fe admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:23.231320 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1287/2562] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:55:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:55:23.235962 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d7ed5d-fb7d-469e-9dcb-e7c0b31af978 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:23.244287 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:55:23.251514 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d7ed5d-fb7d-469e-9dcb-e7c0b31af978 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:23.258138 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb92e1f-47c0-4936-82c0-d1f75d647431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:23.258765 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb92e1f-47c0-4936-82c0-d1f75d647431 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:23.259815 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb92e1f-47c0-4936-82c0-d1f75d647431 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:23.260763 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb92e1f-47c0-4936-82c0-d1f75d647431 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:23.261659 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb92e1f-47c0-4936-82c0-d1f75d647431 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:23.263674 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb92e1f-47c0-4936-82c0-d1f75d647431 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:23.280449 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1276/2563] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:55:23 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:55:23.282206 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264339cd-8b16-42f7-b46c-e8889e8c04fe admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:23.298977 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264339cd-8b16-42f7-b46c-e8889e8c04fe admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:23.316973 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06460ed8-8f51-4a24-a8f5-6a4d5a253df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:23.318047 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06460ed8-8f51-4a24-a8f5-6a4d5a253df4 admin admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:23.318560 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06460ed8-8f51-4a24-a8f5-6a4d5a253df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:23.319020 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06460ed8-8f51-4a24-a8f5-6a4d5a253df4 admin admin] PATH_INFO: `/v3/projects/4a79ec68accc45d0b20a6a3f1b98db42/users/bd9894f9d5bf4bedb5a6926684d376fc/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:23.336684 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06460ed8-8f51-4a24-a8f5-6a4d5a253df4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a79ec68accc45d0b20a6a3f1b98db42, user_id=bd9894f9d5bf4bedb5a6926684d376fc, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:23.340122 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06460ed8-8f51-4a24-a8f5-6a4d5a253df4 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:23.360215 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1288/2564] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:55:23 2022] PUT /identity/v3/projects/4a79ec68accc45d0b20a6a3f1b98db42/users/bd9894f9d5bf4bedb5a6926684d376fc/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 10:55:23.372341 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ab254-2bd8-4863-81bd-acc79a3f0775 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:23.372679 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ab254-2bd8-4863-81bd-acc79a3f0775 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:23.372889 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ab254-2bd8-4863-81bd-acc79a3f0775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:23.405045 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-d39ab254-2bd8-4863-81bd-acc79a3f0775 None None] MFA Rules not processed for user `bd9894f9d5bf4bedb5a6926684d376fc`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:55:23.406322 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-d39ab254-2bd8-4863-81bd-acc79a3f0775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:55:23.484195 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1277/2565] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:55:23 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:55:23.488638 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06460ed8-8f51-4a24-a8f5-6a4d5a253df4 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:23.563393 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06460ed8-8f51-4a24-a8f5-6a4d5a253df4 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:23.569347 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f08be11-d323-4c45-81e1-18415c2c369a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:23.570050 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f08be11-d323-4c45-81e1-18415c2c369a admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:23.570415 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f08be11-d323-4c45-81e1-18415c2c369a admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:23.570763 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f08be11-d323-4c45-81e1-18415c2c369a admin admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:23.571866 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f08be11-d323-4c45-81e1-18415c2c369a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:23.574100 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f08be11-d323-4c45-81e1-18415c2c369a admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:23.578968 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:55:23.605470 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1289/2566] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:55:23 2022] POST /identity/v3/projects => generated 387 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:55:23.609417 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39ab254-2bd8-4863-81bd-acc79a3f0775 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:23.620779 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39ab254-2bd8-4863-81bd-acc79a3f0775 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:23.677549 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b1bf40-18fc-4db5-811a-2d3433f88848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:23.678193 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b1bf40-18fc-4db5-811a-2d3433f88848 admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:23.678475 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b1bf40-18fc-4db5-811a-2d3433f88848 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:23.678747 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b1bf40-18fc-4db5-811a-2d3433f88848 admin admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:23.679381 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b1bf40-18fc-4db5-811a-2d3433f88848 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:23.682117 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b1bf40-18fc-4db5-811a-2d3433f88848 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:23.707324 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1278/2567] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:55:23 2022] POST /identity/v3/users => generated 300 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:55:23.709261 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f08be11-d323-4c45-81e1-18415c2c369a admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:23.720565 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f08be11-d323-4c45-81e1-18415c2c369a admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:23.725330 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e223c89-200d-41ff-a8f5-ecf7d1789600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:23.725944 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e223c89-200d-41ff-a8f5-ecf7d1789600 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:23.726227 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e223c89-200d-41ff-a8f5-ecf7d1789600 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:23.726504 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e223c89-200d-41ff-a8f5-ecf7d1789600 admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:23.727174 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e223c89-200d-41ff-a8f5-ecf7d1789600 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:23.729344 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e223c89-200d-41ff-a8f5-ecf7d1789600 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:23.742824 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1290/2568] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:55:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:55:23.744865 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b1bf40-18fc-4db5-811a-2d3433f88848 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:23.755083 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b1bf40-18fc-4db5-811a-2d3433f88848 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:23.760340 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907cf259-797c-44c5-9fae-de9be08d5b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:23.760998 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907cf259-797c-44c5-9fae-de9be08d5b5c admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:23.761307 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907cf259-797c-44c5-9fae-de9be08d5b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:23.761577 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907cf259-797c-44c5-9fae-de9be08d5b5c admin admin] PATH_INFO: `/v3/projects/d409c292311c4e94a75f207c321cebc2/users/790d5f27ce7e41c280000c7d9e17bfa8/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:23.776885 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907cf259-797c-44c5-9fae-de9be08d5b5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d409c292311c4e94a75f207c321cebc2, user_id=790d5f27ce7e41c280000c7d9e17bfa8, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:23.779633 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907cf259-797c-44c5-9fae-de9be08d5b5c admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:23.794992 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1279/2569] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:55:23 2022] PUT /identity/v3/projects/d409c292311c4e94a75f207c321cebc2/users/790d5f27ce7e41c280000c7d9e17bfa8/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:55:23.798981 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e223c89-200d-41ff-a8f5-ecf7d1789600 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:23.811053 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e223c89-200d-41ff-a8f5-ecf7d1789600 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:23.912054 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad2cf08-1e74-4add-9cec-f94a51009d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:23.913003 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2cf08-1e74-4add-9cec-f94a51009d5a admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:23.913287 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2cf08-1e74-4add-9cec-f94a51009d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:23.913553 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2cf08-1e74-4add-9cec-f94a51009d5a admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:23.914348 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad2cf08-1e74-4add-9cec-f94a51009d5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:23.916331 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad2cf08-1e74-4add-9cec-f94a51009d5a admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:23.931493 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1291/2570] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:55:23 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:55:23.933448 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907cf259-797c-44c5-9fae-de9be08d5b5c admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:23.948897 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907cf259-797c-44c5-9fae-de9be08d5b5c admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:23.953929 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac32e9bf-6355-4196-a1f1-15f6bb534193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:23.954597 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac32e9bf-6355-4196-a1f1-15f6bb534193 admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:23.954880 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac32e9bf-6355-4196-a1f1-15f6bb534193 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:23.955152 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac32e9bf-6355-4196-a1f1-15f6bb534193 admin admin] PATH_INFO: `/v3/projects/d409c292311c4e94a75f207c321cebc2/users/790d5f27ce7e41c280000c7d9e17bfa8/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:23.959476 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac32e9bf-6355-4196-a1f1-15f6bb534193 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d409c292311c4e94a75f207c321cebc2, user_id=790d5f27ce7e41c280000c7d9e17bfa8, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:23.960881 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac32e9bf-6355-4196-a1f1-15f6bb534193 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:23.973524 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1280/2571] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:55:23 2022] PUT /identity/v3/projects/d409c292311c4e94a75f207c321cebc2/users/790d5f27ce7e41c280000c7d9e17bfa8/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 10:55:23.984622 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87db925-acb4-4c9e-896b-08ed7ec98584 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:23.985011 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87db925-acb4-4c9e-896b-08ed7ec98584 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:23.985174 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87db925-acb4-4c9e-896b-08ed7ec98584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:24.028956 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-b87db925-acb4-4c9e-896b-08ed7ec98584 None None] MFA Rules not processed for user `790d5f27ce7e41c280000c7d9e17bfa8`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:55:24.030305 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-b87db925-acb4-4c9e-896b-08ed7ec98584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:55:24.114939 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1292/2572] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:55:23 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:55:24.126662 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac32e9bf-6355-4196-a1f1-15f6bb534193 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:24.128438 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-ac32e9bf-6355-4196-a1f1-15f6bb534193 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:55:24.179529 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:55:24.204476 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac32e9bf-6355-4196-a1f1-15f6bb534193 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:24.211334 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2834f65-7db5-42c0-8d08-06eb1c733734 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:24.212043 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2834f65-7db5-42c0-8d08-06eb1c733734 service cinder] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:24.213649 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2834f65-7db5-42c0-8d08-06eb1c733734 service cinder] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:24.213942 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2834f65-7db5-42c0-8d08-06eb1c733734 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:24.224482 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2834f65-7db5-42c0-8d08-06eb1c733734 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:24.226071 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2834f65-7db5-42c0-8d08-06eb1c733734 service cinder] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:24.247552 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1281/2573] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:55:24 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:55:24.408915 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87db925-acb4-4c9e-896b-08ed7ec98584 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:24.472752 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:55:24.480241 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87db925-acb4-4c9e-896b-08ed7ec98584 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:24.485134 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e198c7-cd72-44bd-9380-4d8a46c0f869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:24.485774 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e198c7-cd72-44bd-9380-4d8a46c0f869 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:24.486057 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e198c7-cd72-44bd-9380-4d8a46c0f869 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:24.486330 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e198c7-cd72-44bd-9380-4d8a46c0f869 admin admin] PATH_INFO: `/v3/users/bd9894f9d5bf4bedb5a6926684d376fc` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:24.487488 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e198c7-cd72-44bd-9380-4d8a46c0f869 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd9894f9d5bf4bedb5a6926684d376fc)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:24.489427 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e198c7-cd72-44bd-9380-4d8a46c0f869 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:24.534955 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-a5e198c7-cd72-44bd-9380-4d8a46c0f869 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd9894f9d5bf4bedb5a6926684d376fc'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:24.542685 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-a5e198c7-cd72-44bd-9380-4d8a46c0f869 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd9894f9d5bf4bedb5a6926684d376fc'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:24.543899 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-a5e198c7-cd72-44bd-9380-4d8a46c0f869 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd9894f9d5bf4bedb5a6926684d376fc'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:24.544510 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-a5e198c7-cd72-44bd-9380-4d8a46c0f869 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd9894f9d5bf4bedb5a6926684d376fc'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:24.557616 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-a5e198c7-cd72-44bd-9380-4d8a46c0f869 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd9894f9d5bf4bedb5a6926684d376fc'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:24.570057 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1293/2574] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:55:24 2022] DELETE /identity/v3/users/bd9894f9d5bf4bedb5a6926684d376fc => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:55:24.574868 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca2f92-5caa-4fc8-a56d-971ecc788cbc None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:24.575306 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca2f92-5caa-4fc8-a56d-971ecc788cbc None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:24.575682 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca2f92-5caa-4fc8-a56d-971ecc788cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:24.596444 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-01ca2f92-5caa-4fc8-a56d-971ecc788cbc None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:55:24.597745 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-01ca2f92-5caa-4fc8-a56d-971ecc788cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:55:24.659548 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1282/2575] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:55:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:55:24.670639 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e198c7-cd72-44bd-9380-4d8a46c0f869 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:24.672185 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-a5e198c7-cd72-44bd-9380-4d8a46c0f869 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:55:24.735689 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e198c7-cd72-44bd-9380-4d8a46c0f869 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:24.737115 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452d7221-d978-4eef-b851-1f8b36765a03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:24.737657 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452d7221-d978-4eef-b851-1f8b36765a03 service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:24.737944 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452d7221-d978-4eef-b851-1f8b36765a03 service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:24.738217 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452d7221-d978-4eef-b851-1f8b36765a03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:24.748803 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452d7221-d978-4eef-b851-1f8b36765a03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:24.750257 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452d7221-d978-4eef-b851-1f8b36765a03 service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:24.760258 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1294/2576] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:55:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:55:25.109641 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ca2f92-5caa-4fc8-a56d-971ecc788cbc None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:25.115616 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-01ca2f92-5caa-4fc8-a56d-971ecc788cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:55:25.137222 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ca2f92-5caa-4fc8-a56d-971ecc788cbc None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:25.147282 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e1e616-37a9-43b0-ad23-cacce0e5d94e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:25.148090 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e1e616-37a9-43b0-ad23-cacce0e5d94e admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:25.148447 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e1e616-37a9-43b0-ad23-cacce0e5d94e admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:25.148816 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e1e616-37a9-43b0-ad23-cacce0e5d94e admin admin] PATH_INFO: `/v3/projects/4a79ec68accc45d0b20a6a3f1b98db42` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:25.150208 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e1e616-37a9-43b0-ad23-cacce0e5d94e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a79ec68accc45d0b20a6a3f1b98db42)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:25.152525 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e1e616-37a9-43b0-ad23-cacce0e5d94e admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:25.203582 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-21e1e616-37a9-43b0-ad23-cacce0e5d94e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a79ec68accc45d0b20a6a3f1b98db42', 'request_id': 'req-21e1e616-37a9-43b0-ad23-cacce0e5d94e'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:25.210071 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-21e1e616-37a9-43b0-ad23-cacce0e5d94e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a79ec68accc45d0b20a6a3f1b98db42', 'request_id': 'req-21e1e616-37a9-43b0-ad23-cacce0e5d94e'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:25.218662 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-21e1e616-37a9-43b0-ad23-cacce0e5d94e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a79ec68accc45d0b20a6a3f1b98db42', 'request_id': 'req-21e1e616-37a9-43b0-ad23-cacce0e5d94e'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:25.230334 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-21e1e616-37a9-43b0-ad23-cacce0e5d94e admin admin] The token cache is being invalidate because project 4a79ec68accc45d0b20a6a3f1b98db42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90653) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:55:25.230645 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-21e1e616-37a9-43b0-ad23-cacce0e5d94e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:25.231291 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-21e1e616-37a9-43b0-ad23-cacce0e5d94e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:25.232848 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1283/2577] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:55:25 2022] DELETE /identity/v3/projects/4a79ec68accc45d0b20a6a3f1b98db42 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:55:25.237158 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452d7221-d978-4eef-b851-1f8b36765a03 service neutron] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:25.248576 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452d7221-d978-4eef-b851-1f8b36765a03 service neutron] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:25.314509 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4e6014-8bdf-4999-8ba6-0ae71e070729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:25.315222 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e6014-8bdf-4999-8ba6-0ae71e070729 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:25.315607 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e6014-8bdf-4999-8ba6-0ae71e070729 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:25.315955 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e6014-8bdf-4999-8ba6-0ae71e070729 admin admin] PATH_INFO: `/v3/users/790d5f27ce7e41c280000c7d9e17bfa8` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:25.317234 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4e6014-8bdf-4999-8ba6-0ae71e070729 admin admin] RBAC: Authorizing `identity:delete_user(user_id=790d5f27ce7e41c280000c7d9e17bfa8)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:25.319147 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4e6014-8bdf-4999-8ba6-0ae71e070729 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:25.341754 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:55:25.371158 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-2d4e6014-8bdf-4999-8ba6-0ae71e070729 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '790d5f27ce7e41c280000c7d9e17bfa8'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:25.379296 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-2d4e6014-8bdf-4999-8ba6-0ae71e070729 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '790d5f27ce7e41c280000c7d9e17bfa8'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:25.380056 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-2d4e6014-8bdf-4999-8ba6-0ae71e070729 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '790d5f27ce7e41c280000c7d9e17bfa8'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:25.380660 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-2d4e6014-8bdf-4999-8ba6-0ae71e070729 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '790d5f27ce7e41c280000c7d9e17bfa8'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:25.393988 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-2d4e6014-8bdf-4999-8ba6-0ae71e070729 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '790d5f27ce7e41c280000c7d9e17bfa8'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:25.408444 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1295/2578] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:55:25 2022] DELETE /identity/v3/users/790d5f27ce7e41c280000c7d9e17bfa8 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 10:55:25.687765 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e1e616-37a9-43b0-ad23-cacce0e5d94e admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:25.690011 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-21e1e616-37a9-43b0-ad23-cacce0e5d94e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:55:25.705113 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e1e616-37a9-43b0-ad23-cacce0e5d94e admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:25.769472 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628ea930-78dc-4228-b198-25ab9026eace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:25.770116 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628ea930-78dc-4228-b198-25ab9026eace admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:25.770397 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628ea930-78dc-4228-b198-25ab9026eace admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:25.770667 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628ea930-78dc-4228-b198-25ab9026eace admin admin] PATH_INFO: `/v3/projects/d409c292311c4e94a75f207c321cebc2` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:25.771871 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628ea930-78dc-4228-b198-25ab9026eace admin admin] RBAC: Authorizing `identity:delete_project(project_id=d409c292311c4e94a75f207c321cebc2)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:25.774039 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628ea930-78dc-4228-b198-25ab9026eace admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:25.776459 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:55:25.948630 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-628ea930-78dc-4228-b198-25ab9026eace admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd409c292311c4e94a75f207c321cebc2', 'request_id': 'req-628ea930-78dc-4228-b198-25ab9026eace'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:25.955036 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-628ea930-78dc-4228-b198-25ab9026eace admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd409c292311c4e94a75f207c321cebc2', 'request_id': 'req-628ea930-78dc-4228-b198-25ab9026eace'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:25.963806 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-628ea930-78dc-4228-b198-25ab9026eace admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd409c292311c4e94a75f207c321cebc2', 'request_id': 'req-628ea930-78dc-4228-b198-25ab9026eace'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:25.975787 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-628ea930-78dc-4228-b198-25ab9026eace admin admin] The token cache is being invalidate because project d409c292311c4e94a75f207c321cebc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90653) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:55:25.976079 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-628ea930-78dc-4228-b198-25ab9026eace admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:25.976857 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-628ea930-78dc-4228-b198-25ab9026eace admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:25.978699 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1284/2579] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:55:25 2022] DELETE /identity/v3/projects/d409c292311c4e94a75f207c321cebc2 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:55:25.983598 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4766b9a-4af0-44aa-b698-180240aa2ea7 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:25.983895 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4766b9a-4af0-44aa-b698-180240aa2ea7 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:25.984444 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4766b9a-4af0-44aa-b698-180240aa2ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:26.004175 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-b4766b9a-4af0-44aa-b698-180240aa2ea7 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:55:26.006513 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-b4766b9a-4af0-44aa-b698-180240aa2ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:55:26.073226 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1296/2580] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:55:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:55:26.081537 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f69ecfb-1bb5-4c29-b96c-e5101744e941 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:26.081816 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f69ecfb-1bb5-4c29-b96c-e5101744e941 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:26.082247 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f69ecfb-1bb5-4c29-b96c-e5101744e941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:26.108710 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-2f69ecfb-1bb5-4c29-b96c-e5101744e941 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:55:26.110187 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-2f69ecfb-1bb5-4c29-b96c-e5101744e941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:55:26.118954 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1285/2581] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:55:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:55:26.120789 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaf2335-ff55-4719-af86-c27f96b7597c None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:26.121541 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaf2335-ff55-4719-af86-c27f96b7597c None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:26.121541 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaf2335-ff55-4719-af86-c27f96b7597c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:26.124828 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f69ecfb-1bb5-4c29-b96c-e5101744e941 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:26.135560 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f69ecfb-1bb5-4c29-b96c-e5101744e941 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:26.140855 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b80a8a-3cbc-493e-921e-ff147cd373c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:26.141437 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b80a8a-3cbc-493e-921e-ff147cd373c2 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:26.141713 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b80a8a-3cbc-493e-921e-ff147cd373c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:26.141980 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b80a8a-3cbc-493e-921e-ff147cd373c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:26.142616 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b80a8a-3cbc-493e-921e-ff147cd373c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:55:26.142914 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b80a8a-3cbc-493e-921e-ff147cd373c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:26.143074 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-aeaf2335-ff55-4719-af86-c27f96b7597c None None] MFA Rules not processed for user `8d0af2b5f354439d92bfa317a0baefd5`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:55:26.144404 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-aeaf2335-ff55-4719-af86-c27f96b7597c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:55:26.144561 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b80a8a-3cbc-493e-921e-ff147cd373c2 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:26.158560 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1286/2582] 10.222.0.15 () {58 vars in 1153 bytes} [Sun Sep 11 10:55:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:55:26.163109 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b80a8a-3cbc-493e-921e-ff147cd373c2 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:26.176203 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b80a8a-3cbc-493e-921e-ff147cd373c2 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:26.183009 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d6338e-a41c-4d69-ad9a-56dfeb1df27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:26.183645 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d6338e-a41c-4d69-ad9a-56dfeb1df27f admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:26.183924 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d6338e-a41c-4d69-ad9a-56dfeb1df27f admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:26.184198 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d6338e-a41c-4d69-ad9a-56dfeb1df27f admin admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:26.185314 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d6338e-a41c-4d69-ad9a-56dfeb1df27f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:26.188233 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d6338e-a41c-4d69-ad9a-56dfeb1df27f admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:26.206070 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1297/2583] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:55:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:55:26.222764 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1287/2584] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:55:26 2022] POST /identity/v3/projects => generated 379 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 10:55:26.226766 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaf2335-ff55-4719-af86-c27f96b7597c None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:26.238527 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaf2335-ff55-4719-af86-c27f96b7597c None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:26.283366 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:55:26.306421 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ec7ef-9f3f-478f-aa26-88bc23d384f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:26.307163 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ec7ef-9f3f-478f-aa26-88bc23d384f5 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:26.307693 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ec7ef-9f3f-478f-aa26-88bc23d384f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:26.307871 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ec7ef-9f3f-478f-aa26-88bc23d384f5 admin admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:26.308698 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960ec7ef-9f3f-478f-aa26-88bc23d384f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:26.310671 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960ec7ef-9f3f-478f-aa26-88bc23d384f5 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:26.346301 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1298/2585] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:55:26 2022] POST /identity/v3/users => generated 290 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:55:26.348395 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d6338e-a41c-4d69-ad9a-56dfeb1df27f admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:26.362404 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d6338e-a41c-4d69-ad9a-56dfeb1df27f admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:26.367694 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b12ab1-dbc7-44b6-90f4-09cca9f47df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:26.368952 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b12ab1-dbc7-44b6-90f4-09cca9f47df8 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:26.369452 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b12ab1-dbc7-44b6-90f4-09cca9f47df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:26.369732 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b12ab1-dbc7-44b6-90f4-09cca9f47df8 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:26.370364 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b12ab1-dbc7-44b6-90f4-09cca9f47df8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:26.372025 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b12ab1-dbc7-44b6-90f4-09cca9f47df8 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:26.382252 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1288/2586] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:55:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:55:26.387207 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ec7ef-9f3f-478f-aa26-88bc23d384f5 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:26.398072 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ec7ef-9f3f-478f-aa26-88bc23d384f5 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:26.403628 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f799cf-ed5d-454f-ad9b-4c1dc9eea35d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:26.404245 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f799cf-ed5d-454f-ad9b-4c1dc9eea35d admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:26.404604 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f799cf-ed5d-454f-ad9b-4c1dc9eea35d admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:26.404863 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f799cf-ed5d-454f-ad9b-4c1dc9eea35d admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:26.405587 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f799cf-ed5d-454f-ad9b-4c1dc9eea35d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:26.407171 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f799cf-ed5d-454f-ad9b-4c1dc9eea35d admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:26.420729 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1299/2587] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:55:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 10:55:26.422110 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b12ab1-dbc7-44b6-90f4-09cca9f47df8 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:26.432576 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b12ab1-dbc7-44b6-90f4-09cca9f47df8 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:26.437151 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1e07e4-e01a-4562-a099-dc36828c3f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:26.437798 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e07e4-e01a-4562-a099-dc36828c3f6c admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:26.438076 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e07e4-e01a-4562-a099-dc36828c3f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:26.438346 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e07e4-e01a-4562-a099-dc36828c3f6c admin admin] PATH_INFO: `/v3/projects/d40c2c6addad42ffa9daaedefb45535a/users/860d7e8b8ec0459b9e9f66693252f3ee/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:26.453907 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1e07e4-e01a-4562-a099-dc36828c3f6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d40c2c6addad42ffa9daaedefb45535a, user_id=860d7e8b8ec0459b9e9f66693252f3ee, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:26.456690 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1e07e4-e01a-4562-a099-dc36828c3f6c admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:26.469693 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1289/2588] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:55:26 2022] PUT /identity/v3/projects/d40c2c6addad42ffa9daaedefb45535a/users/860d7e8b8ec0459b9e9f66693252f3ee/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:55:26.536986 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10030b4-5ada-4972-adad-0fcb9ef17741 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:26.537265 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10030b4-5ada-4972-adad-0fcb9ef17741 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:26.537539 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10030b4-5ada-4972-adad-0fcb9ef17741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:26.566537 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-e10030b4-5ada-4972-adad-0fcb9ef17741 None None] MFA Rules not processed for user `860d7e8b8ec0459b9e9f66693252f3ee`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:55:26.567812 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-e10030b4-5ada-4972-adad-0fcb9ef17741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:55:26.632614 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1300/2589] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:55:26 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:55:26.637349 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1e07e4-e01a-4562-a099-dc36828c3f6c admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:26.699205 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1e07e4-e01a-4562-a099-dc36828c3f6c admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:26.704000 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b4594d-59b7-4711-ae25-da3c8c05c4e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:26.704607 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b4594d-59b7-4711-ae25-da3c8c05c4e5 admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:26.704890 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b4594d-59b7-4711-ae25-da3c8c05c4e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:26.705161 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b4594d-59b7-4711-ae25-da3c8c05c4e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:26.705796 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b4594d-59b7-4711-ae25-da3c8c05c4e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:26.707881 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b4594d-59b7-4711-ae25-da3c8c05c4e5 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:26.730672 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:55:26.744706 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1290/2590] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:55:26 2022] POST /identity/v3/projects => generated 379 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:55:26.746452 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10030b4-5ada-4972-adad-0fcb9ef17741 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:26.760111 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10030b4-5ada-4972-adad-0fcb9ef17741 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:26.820669 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b799806-f8af-445c-9a14-e29e842024c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:26.822012 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b799806-f8af-445c-9a14-e29e842024c9 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:26.822012 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b799806-f8af-445c-9a14-e29e842024c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:26.822325 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b799806-f8af-445c-9a14-e29e842024c9 admin admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:26.823257 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b799806-f8af-445c-9a14-e29e842024c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:26.825179 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b799806-f8af-445c-9a14-e29e842024c9 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:26.850469 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1301/2591] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:55:26 2022] POST /identity/v3/users => generated 296 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:55:26.854895 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b4594d-59b7-4711-ae25-da3c8c05c4e5 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:26.868529 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b4594d-59b7-4711-ae25-da3c8c05c4e5 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:26.873275 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2d2b2a-8f11-4da6-bef0-0517a418fffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:26.873877 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2d2b2a-8f11-4da6-bef0-0517a418fffd admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:26.874153 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2d2b2a-8f11-4da6-bef0-0517a418fffd admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:26.874423 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2d2b2a-8f11-4da6-bef0-0517a418fffd admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:26.875090 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2d2b2a-8f11-4da6-bef0-0517a418fffd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:26.876691 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2d2b2a-8f11-4da6-bef0-0517a418fffd admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:26.887524 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1291/2592] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:55:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:55:26.897732 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b799806-f8af-445c-9a14-e29e842024c9 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:26.909231 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b799806-f8af-445c-9a14-e29e842024c9 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:26.914408 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2839e12f-65bc-49a3-9f6b-f5e83d5c287b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:26.916138 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2839e12f-65bc-49a3-9f6b-f5e83d5c287b admin admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:26.916499 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2839e12f-65bc-49a3-9f6b-f5e83d5c287b admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:26.916787 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2839e12f-65bc-49a3-9f6b-f5e83d5c287b admin admin] PATH_INFO: `/v3/projects/9b090ba9a42541179647fa71f253436f/users/f160261e9c184dc89e0a4c79943431e5/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:26.936922 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2839e12f-65bc-49a3-9f6b-f5e83d5c287b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b090ba9a42541179647fa71f253436f, user_id=f160261e9c184dc89e0a4c79943431e5, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:26.939705 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2839e12f-65bc-49a3-9f6b-f5e83d5c287b admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:26.959563 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1302/2593] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:55:26 2022] PUT /identity/v3/projects/9b090ba9a42541179647fa71f253436f/users/f160261e9c184dc89e0a4c79943431e5/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 10:55:26.963172 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2d2b2a-8f11-4da6-bef0-0517a418fffd admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:26.973783 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2d2b2a-8f11-4da6-bef0-0517a418fffd admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:27.033222 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a5799a-41ab-4622-9007-cbfc03a5b54c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:27.033825 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a5799a-41ab-4622-9007-cbfc03a5b54c admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:27.034104 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a5799a-41ab-4622-9007-cbfc03a5b54c admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:27.034373 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a5799a-41ab-4622-9007-cbfc03a5b54c admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:27.034998 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a5799a-41ab-4622-9007-cbfc03a5b54c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:27.036602 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a5799a-41ab-4622-9007-cbfc03a5b54c admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:27.050684 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1292/2594] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:55:26 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:55:27.052156 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2839e12f-65bc-49a3-9f6b-f5e83d5c287b admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:27.062336 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2839e12f-65bc-49a3-9f6b-f5e83d5c287b admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:27.067210 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea57a2-8ee9-46e8-9ff1-9de8d590f9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:27.067839 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea57a2-8ee9-46e8-9ff1-9de8d590f9b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:27.068126 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea57a2-8ee9-46e8-9ff1-9de8d590f9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:27.068399 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea57a2-8ee9-46e8-9ff1-9de8d590f9b3 admin admin] PATH_INFO: `/v3/projects/9b090ba9a42541179647fa71f253436f/users/f160261e9c184dc89e0a4c79943431e5/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:27.070417 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ea57a2-8ee9-46e8-9ff1-9de8d590f9b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b090ba9a42541179647fa71f253436f, user_id=f160261e9c184dc89e0a4c79943431e5, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:27.073247 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ea57a2-8ee9-46e8-9ff1-9de8d590f9b3 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:27.083729 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1303/2595] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:55:27 2022] PUT /identity/v3/projects/9b090ba9a42541179647fa71f253436f/users/f160261e9c184dc89e0a4c79943431e5/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:55:27.094286 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8faf2-a668-4397-82d1-2816f2405ba8 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:27.094983 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8faf2-a668-4397-82d1-2816f2405ba8 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:27.095362 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8faf2-a668-4397-82d1-2816f2405ba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:27.127175 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-b2d8faf2-a668-4397-82d1-2816f2405ba8 None None] MFA Rules not processed for user `f160261e9c184dc89e0a4c79943431e5`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:55:27.129739 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-b2d8faf2-a668-4397-82d1-2816f2405ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:55:27.204931 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1293/2596] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:55:27 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:55:27.217237 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea57a2-8ee9-46e8-9ff1-9de8d590f9b3 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:27.218920 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-10ea57a2-8ee9-46e8-9ff1-9de8d590f9b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:55:27.224500 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:55:27.284912 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea57a2-8ee9-46e8-9ff1-9de8d590f9b3 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:27.288897 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73ff03e-1454-4561-aa1a-ff287e97f336 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:27.289512 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ff03e-1454-4561-aa1a-ff287e97f336 service cinder] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:27.289798 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ff03e-1454-4561-aa1a-ff287e97f336 service cinder] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:27.290111 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ff03e-1454-4561-aa1a-ff287e97f336 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:27.300443 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73ff03e-1454-4561-aa1a-ff287e97f336 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:27.301886 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73ff03e-1454-4561-aa1a-ff287e97f336 service cinder] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:27.312212 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1304/2597] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:55:27 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:55:27.735203 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d8faf2-a668-4397-82d1-2816f2405ba8 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:27.796510 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d8faf2-a668-4397-82d1-2816f2405ba8 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:27.801020 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94400850-ede7-4b74-9b53-d840c0f16a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:27.801640 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94400850-ede7-4b74-9b53-d840c0f16a2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:27.801920 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94400850-ede7-4b74-9b53-d840c0f16a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:27.802195 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94400850-ede7-4b74-9b53-d840c0f16a2d admin admin] PATH_INFO: `/v3/users/860d7e8b8ec0459b9e9f66693252f3ee` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:27.803300 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94400850-ede7-4b74-9b53-d840c0f16a2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=860d7e8b8ec0459b9e9f66693252f3ee)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:27.805248 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94400850-ede7-4b74-9b53-d840c0f16a2d admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:27.820085 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:55:27.854549 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-94400850-ede7-4b74-9b53-d840c0f16a2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '860d7e8b8ec0459b9e9f66693252f3ee'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:27.861940 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-94400850-ede7-4b74-9b53-d840c0f16a2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '860d7e8b8ec0459b9e9f66693252f3ee'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:27.862620 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-94400850-ede7-4b74-9b53-d840c0f16a2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '860d7e8b8ec0459b9e9f66693252f3ee'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:27.863239 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-94400850-ede7-4b74-9b53-d840c0f16a2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '860d7e8b8ec0459b9e9f66693252f3ee'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:27.871256 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-94400850-ede7-4b74-9b53-d840c0f16a2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '860d7e8b8ec0459b9e9f66693252f3ee'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:27.890197 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1294/2598] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:55:27 2022] DELETE /identity/v3/users/860d7e8b8ec0459b9e9f66693252f3ee => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:55:27.894544 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f334f410-bcc0-4b4c-87d3-35eeb46743d5 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:27.894827 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f334f410-bcc0-4b4c-87d3-35eeb46743d5 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:27.895098 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f334f410-bcc0-4b4c-87d3-35eeb46743d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:27.914352 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-f334f410-bcc0-4b4c-87d3-35eeb46743d5 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:55:27.915833 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-f334f410-bcc0-4b4c-87d3-35eeb46743d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:55:27.978906 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1305/2599] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:55:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:55:27.990138 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94400850-ede7-4b74-9b53-d840c0f16a2d admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:27.991821 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-94400850-ede7-4b74-9b53-d840c0f16a2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:55:28.059936 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94400850-ede7-4b74-9b53-d840c0f16a2d admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:28.064112 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1ec5a4-b4ff-4dee-9e03-bbb45f49ed76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:28.064726 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1ec5a4-b4ff-4dee-9e03-bbb45f49ed76 service neutron] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:28.065010 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1ec5a4-b4ff-4dee-9e03-bbb45f49ed76 service neutron] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:28.065280 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1ec5a4-b4ff-4dee-9e03-bbb45f49ed76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:28.077332 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1ec5a4-b4ff-4dee-9e03-bbb45f49ed76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:28.078880 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1ec5a4-b4ff-4dee-9e03-bbb45f49ed76 service neutron] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:28.091081 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1295/2600] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:55:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:55:28.449566 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f334f410-bcc0-4b4c-87d3-35eeb46743d5 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:28.451104 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-f334f410-bcc0-4b4c-87d3-35eeb46743d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:55:28.489572 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f334f410-bcc0-4b4c-87d3-35eeb46743d5 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:28.494969 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b366a8da-0370-4aad-add9-b44d037cf3d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:28.495663 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b366a8da-0370-4aad-add9-b44d037cf3d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:28.495944 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b366a8da-0370-4aad-add9-b44d037cf3d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:28.496219 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b366a8da-0370-4aad-add9-b44d037cf3d4 admin admin] PATH_INFO: `/v3/projects/d40c2c6addad42ffa9daaedefb45535a` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:28.497580 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b366a8da-0370-4aad-add9-b44d037cf3d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d40c2c6addad42ffa9daaedefb45535a)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:28.499510 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b366a8da-0370-4aad-add9-b44d037cf3d4 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:28.549098 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-b366a8da-0370-4aad-add9-b44d037cf3d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd40c2c6addad42ffa9daaedefb45535a', 'request_id': 'req-b366a8da-0370-4aad-add9-b44d037cf3d4'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:28.555616 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-b366a8da-0370-4aad-add9-b44d037cf3d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd40c2c6addad42ffa9daaedefb45535a', 'request_id': 'req-b366a8da-0370-4aad-add9-b44d037cf3d4'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:28.561283 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-b366a8da-0370-4aad-add9-b44d037cf3d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd40c2c6addad42ffa9daaedefb45535a', 'request_id': 'req-b366a8da-0370-4aad-add9-b44d037cf3d4'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:28.573717 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-b366a8da-0370-4aad-add9-b44d037cf3d4 admin admin] The token cache is being invalidate because project d40c2c6addad42ffa9daaedefb45535a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:55:28.574019 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-b366a8da-0370-4aad-add9-b44d037cf3d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:28.574682 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-b366a8da-0370-4aad-add9-b44d037cf3d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:28.576217 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1306/2601] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:55:28 2022] DELETE /identity/v3/projects/d40c2c6addad42ffa9daaedefb45535a => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:55:28.580337 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1ec5a4-b4ff-4dee-9e03-bbb45f49ed76 service neutron] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:28.590630 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1ec5a4-b4ff-4dee-9e03-bbb45f49ed76 service neutron] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:28.651195 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2427710b-8b5d-4d23-b6b4-515bd444e245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:28.651982 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2427710b-8b5d-4d23-b6b4-515bd444e245 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:28.652261 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2427710b-8b5d-4d23-b6b4-515bd444e245 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:28.652535 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2427710b-8b5d-4d23-b6b4-515bd444e245 admin admin] PATH_INFO: `/v3/users/f160261e9c184dc89e0a4c79943431e5` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:28.653659 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2427710b-8b5d-4d23-b6b4-515bd444e245 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f160261e9c184dc89e0a4c79943431e5)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:28.655621 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2427710b-8b5d-4d23-b6b4-515bd444e245 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:28.693843 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:55:28.704368 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-2427710b-8b5d-4d23-b6b4-515bd444e245 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f160261e9c184dc89e0a4c79943431e5'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:28.712747 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-2427710b-8b5d-4d23-b6b4-515bd444e245 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f160261e9c184dc89e0a4c79943431e5'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:28.713372 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-2427710b-8b5d-4d23-b6b4-515bd444e245 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f160261e9c184dc89e0a4c79943431e5'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:28.713998 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-2427710b-8b5d-4d23-b6b4-515bd444e245 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f160261e9c184dc89e0a4c79943431e5'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:28.721043 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-2427710b-8b5d-4d23-b6b4-515bd444e245 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f160261e9c184dc89e0a4c79943431e5'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:28.738958 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1296/2602] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:55:28 2022] DELETE /identity/v3/users/f160261e9c184dc89e0a4c79943431e5 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 10:55:29.050659 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b366a8da-0370-4aad-add9-b44d037cf3d4 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:29.052284 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-b366a8da-0370-4aad-add9-b44d037cf3d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:55:29.069647 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b366a8da-0370-4aad-add9-b44d037cf3d4 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:29.103517 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:55:29.153452 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1016cc81-a066-4c6d-a50c-99d80d5c55df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:29.154220 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1016cc81-a066-4c6d-a50c-99d80d5c55df admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:29.154568 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1016cc81-a066-4c6d-a50c-99d80d5c55df admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:29.154947 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1016cc81-a066-4c6d-a50c-99d80d5c55df admin admin] PATH_INFO: `/v3/projects/9b090ba9a42541179647fa71f253436f` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:29.156609 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1016cc81-a066-4c6d-a50c-99d80d5c55df admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b090ba9a42541179647fa71f253436f)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:29.158574 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1016cc81-a066-4c6d-a50c-99d80d5c55df admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:29.207167 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-1016cc81-a066-4c6d-a50c-99d80d5c55df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b090ba9a42541179647fa71f253436f', 'request_id': 'req-1016cc81-a066-4c6d-a50c-99d80d5c55df'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:29.213838 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-1016cc81-a066-4c6d-a50c-99d80d5c55df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b090ba9a42541179647fa71f253436f', 'request_id': 'req-1016cc81-a066-4c6d-a50c-99d80d5c55df'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:29.219837 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-1016cc81-a066-4c6d-a50c-99d80d5c55df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b090ba9a42541179647fa71f253436f', 'request_id': 'req-1016cc81-a066-4c6d-a50c-99d80d5c55df'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:29.233527 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-1016cc81-a066-4c6d-a50c-99d80d5c55df admin admin] The token cache is being invalidate because project 9b090ba9a42541179647fa71f253436f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:55:29.233817 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-1016cc81-a066-4c6d-a50c-99d80d5c55df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:29.234951 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-1016cc81-a066-4c6d-a50c-99d80d5c55df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:29.236496 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1307/2603] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:55:29 2022] DELETE /identity/v3/projects/9b090ba9a42541179647fa71f253436f => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 10:55:29.241400 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10324f1-a843-41ad-8ec6-47a4e995a76e None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:29.241621 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10324f1-a843-41ad-8ec6-47a4e995a76e None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:29.241898 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10324f1-a843-41ad-8ec6-47a4e995a76e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:29.262864 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-b10324f1-a843-41ad-8ec6-47a4e995a76e None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:55:29.264282 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-b10324f1-a843-41ad-8ec6-47a4e995a76e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:55:29.331986 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1297/2604] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:55:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 10:55:29.341118 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c30e672-9b0b-4664-8e16-37500bc1ae6f None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:29.341413 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c30e672-9b0b-4664-8e16-37500bc1ae6f None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:29.341719 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c30e672-9b0b-4664-8e16-37500bc1ae6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:29.365246 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-9c30e672-9b0b-4664-8e16-37500bc1ae6f None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:55:29.366435 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-9c30e672-9b0b-4664-8e16-37500bc1ae6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:55:29.376396 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1308/2605] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:55:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:55:29.380700 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10324f1-a843-41ad-8ec6-47a4e995a76e None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:29.390298 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10324f1-a843-41ad-8ec6-47a4e995a76e None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:29.394835 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb59e56-ed3b-4a9b-9447-786df60c4b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:29.395583 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb59e56-ed3b-4a9b-9447-786df60c4b8c admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:29.395723 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb59e56-ed3b-4a9b-9447-786df60c4b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:29.396040 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb59e56-ed3b-4a9b-9447-786df60c4b8c admin admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:29.396765 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb59e56-ed3b-4a9b-9447-786df60c4b8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:55:29.397176 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb59e56-ed3b-4a9b-9447-786df60c4b8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:29.398797 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb59e56-ed3b-4a9b-9447-786df60c4b8c admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:29.411815 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1298/2606] 10.222.0.15 () {58 vars in 1153 bytes} [Sun Sep 11 10:55:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:55:29.416886 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c30e672-9b0b-4664-8e16-37500bc1ae6f None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:29.426010 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c30e672-9b0b-4664-8e16-37500bc1ae6f None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:29.430660 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d225c5c3-3e8f-487e-b665-2f6c0efe0ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:29.431224 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d225c5c3-3e8f-487e-b665-2f6c0efe0ea6 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:29.431531 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d225c5c3-3e8f-487e-b665-2f6c0efe0ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:29.431806 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d225c5c3-3e8f-487e-b665-2f6c0efe0ea6 admin admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:29.432432 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d225c5c3-3e8f-487e-b665-2f6c0efe0ea6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:29.434319 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d225c5c3-3e8f-487e-b665-2f6c0efe0ea6 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:29.462205 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1309/2607] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:55:29 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:55:29.466751 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb59e56-ed3b-4a9b-9447-786df60c4b8c admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:29.476623 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb59e56-ed3b-4a9b-9447-786df60c4b8c admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:29.536808 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3787fec-35cc-4ec7-bda4-672733893873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:29.537700 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3787fec-35cc-4ec7-bda4-672733893873 admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:29.537982 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3787fec-35cc-4ec7-bda4-672733893873 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:29.538252 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3787fec-35cc-4ec7-bda4-672733893873 admin admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:29.539134 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3787fec-35cc-4ec7-bda4-672733893873 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:29.541326 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3787fec-35cc-4ec7-bda4-672733893873 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:29.563296 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1299/2608] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:55:29 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:55:29.567169 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d225c5c3-3e8f-487e-b665-2f6c0efe0ea6 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:29.578538 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d225c5c3-3e8f-487e-b665-2f6c0efe0ea6 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:29.583700 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47006061-6a2b-4e8b-b0eb-3d1928b7ca42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:29.584475 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47006061-6a2b-4e8b-b0eb-3d1928b7ca42 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:29.584756 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47006061-6a2b-4e8b-b0eb-3d1928b7ca42 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:29.585033 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47006061-6a2b-4e8b-b0eb-3d1928b7ca42 admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:29.585730 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47006061-6a2b-4e8b-b0eb-3d1928b7ca42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:29.588015 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47006061-6a2b-4e8b-b0eb-3d1928b7ca42 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:29.598934 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1310/2609] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:55:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:55:29.603226 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3787fec-35cc-4ec7-bda4-672733893873 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:29.613693 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3787fec-35cc-4ec7-bda4-672733893873 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:29.618360 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fee8945-b010-4a31-84b5-dd51d4aad5a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:29.618989 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fee8945-b010-4a31-84b5-dd51d4aad5a7 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:29.619272 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fee8945-b010-4a31-84b5-dd51d4aad5a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:29.619572 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fee8945-b010-4a31-84b5-dd51d4aad5a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:29.620211 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fee8945-b010-4a31-84b5-dd51d4aad5a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:29.621795 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fee8945-b010-4a31-84b5-dd51d4aad5a7 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:29.632482 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1300/2610] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:55:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:55:29.637283 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47006061-6a2b-4e8b-b0eb-3d1928b7ca42 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:29.647503 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47006061-6a2b-4e8b-b0eb-3d1928b7ca42 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:29.653125 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626df68e-13f2-4a05-8eab-16af1aa534cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:29.653819 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626df68e-13f2-4a05-8eab-16af1aa534cc admin admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:29.654099 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626df68e-13f2-4a05-8eab-16af1aa534cc admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:29.654375 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626df68e-13f2-4a05-8eab-16af1aa534cc admin admin] PATH_INFO: `/v3/projects/01fe93ffa5554f24bbfa220b5b0dbb40/users/ff3a246591cb4bcea9e8d8808f8a9575/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:29.669574 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626df68e-13f2-4a05-8eab-16af1aa534cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=01fe93ffa5554f24bbfa220b5b0dbb40, user_id=ff3a246591cb4bcea9e8d8808f8a9575, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:29.672679 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626df68e-13f2-4a05-8eab-16af1aa534cc admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:29.687921 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1311/2611] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:55:29 2022] PUT /identity/v3/projects/01fe93ffa5554f24bbfa220b5b0dbb40/users/ff3a246591cb4bcea9e8d8808f8a9575/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:55:29.700426 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460dfb0a-fb7d-44eb-8211-64d8d13fb813 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:29.700831 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460dfb0a-fb7d-44eb-8211-64d8d13fb813 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:29.700994 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460dfb0a-fb7d-44eb-8211-64d8d13fb813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:29.734570 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-460dfb0a-fb7d-44eb-8211-64d8d13fb813 None None] MFA Rules not processed for user `ff3a246591cb4bcea9e8d8808f8a9575`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:55:29.735876 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-460dfb0a-fb7d-44eb-8211-64d8d13fb813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:55:29.741677 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:55:29.811723 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1301/2612] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:55:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:55:29.816519 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626df68e-13f2-4a05-8eab-16af1aa534cc admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:29.875869 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:55:29.881244 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626df68e-13f2-4a05-8eab-16af1aa534cc admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:29.889391 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19edf20b-3ea1-4088-b83d-06784b233a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:29.890111 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19edf20b-3ea1-4088-b83d-06784b233a9b admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:29.890635 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19edf20b-3ea1-4088-b83d-06784b233a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:29.890985 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19edf20b-3ea1-4088-b83d-06784b233a9b admin admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:29.891819 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19edf20b-3ea1-4088-b83d-06784b233a9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:29.893780 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19edf20b-3ea1-4088-b83d-06784b233a9b admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:29.922551 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1312/2613] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:55:29 2022] POST /identity/v3/projects => generated 371 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 10:55:29.926659 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460dfb0a-fb7d-44eb-8211-64d8d13fb813 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:29.938056 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460dfb0a-fb7d-44eb-8211-64d8d13fb813 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:29.994837 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f607f19-5c91-4311-9100-66da23032b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:29.995474 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f607f19-5c91-4311-9100-66da23032b05 admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:29.995760 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f607f19-5c91-4311-9100-66da23032b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:29.996033 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f607f19-5c91-4311-9100-66da23032b05 admin admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:29.996662 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f607f19-5c91-4311-9100-66da23032b05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:29.998569 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f607f19-5c91-4311-9100-66da23032b05 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:30.023303 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1302/2614] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:55:29 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 10:55:30.024585 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19edf20b-3ea1-4088-b83d-06784b233a9b admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:30.035465 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19edf20b-3ea1-4088-b83d-06784b233a9b admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:30.040420 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e461b81-150c-435a-b61a-eb436c9c2cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:30.041138 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e461b81-150c-435a-b61a-eb436c9c2cb4 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:30.041522 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e461b81-150c-435a-b61a-eb436c9c2cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:30.041864 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e461b81-150c-435a-b61a-eb436c9c2cb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:30.042604 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e461b81-150c-435a-b61a-eb436c9c2cb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:30.044382 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e461b81-150c-435a-b61a-eb436c9c2cb4 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:30.055263 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1313/2615] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:55:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:55:30.060552 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f607f19-5c91-4311-9100-66da23032b05 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:30.070493 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f607f19-5c91-4311-9100-66da23032b05 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:30.075839 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1393f7-cad4-4083-94cb-426330913242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:30.076482 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1393f7-cad4-4083-94cb-426330913242 admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:30.076759 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1393f7-cad4-4083-94cb-426330913242 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:30.077027 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1393f7-cad4-4083-94cb-426330913242 admin admin] PATH_INFO: `/v3/projects/6381f1c748d24833abc3a2cb5ba22368/users/32da9a3fd2d44b34b27c4f166ecc7ecf/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:30.091516 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1393f7-cad4-4083-94cb-426330913242 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6381f1c748d24833abc3a2cb5ba22368, user_id=32da9a3fd2d44b34b27c4f166ecc7ecf, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:30.094124 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1393f7-cad4-4083-94cb-426330913242 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:30.108485 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1303/2616] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:55:30 2022] PUT /identity/v3/projects/6381f1c748d24833abc3a2cb5ba22368/users/32da9a3fd2d44b34b27c4f166ecc7ecf/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:55:30.112576 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e461b81-150c-435a-b61a-eb436c9c2cb4 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:30.131475 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e461b81-150c-435a-b61a-eb436c9c2cb4 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:30.191261 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca494592-606e-4881-bd7e-02b394e63a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:30.191916 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca494592-606e-4881-bd7e-02b394e63a9f admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:30.192201 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca494592-606e-4881-bd7e-02b394e63a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:30.192595 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca494592-606e-4881-bd7e-02b394e63a9f admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:30.193270 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca494592-606e-4881-bd7e-02b394e63a9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:30.194865 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca494592-606e-4881-bd7e-02b394e63a9f admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:30.206388 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1314/2617] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:55:30 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:55:30.209828 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1393f7-cad4-4083-94cb-426330913242 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:30.220728 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1393f7-cad4-4083-94cb-426330913242 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:30.225345 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c371b2-89f2-455a-bdda-fbf55f88f877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:30.225998 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c371b2-89f2-455a-bdda-fbf55f88f877 admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:30.226277 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c371b2-89f2-455a-bdda-fbf55f88f877 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:30.226551 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c371b2-89f2-455a-bdda-fbf55f88f877 admin admin] PATH_INFO: `/v3/projects/6381f1c748d24833abc3a2cb5ba22368/users/32da9a3fd2d44b34b27c4f166ecc7ecf/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:30.228661 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c371b2-89f2-455a-bdda-fbf55f88f877 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6381f1c748d24833abc3a2cb5ba22368, user_id=32da9a3fd2d44b34b27c4f166ecc7ecf, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:30.231276 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c371b2-89f2-455a-bdda-fbf55f88f877 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:30.242030 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1304/2618] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:55:30 2022] PUT /identity/v3/projects/6381f1c748d24833abc3a2cb5ba22368/users/32da9a3fd2d44b34b27c4f166ecc7ecf/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:55:30.253431 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95327fd-ff5c-462e-8a7d-553824965fcd None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:30.253802 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95327fd-ff5c-462e-8a7d-553824965fcd None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:30.254089 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95327fd-ff5c-462e-8a7d-553824965fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:30.283339 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-f95327fd-ff5c-462e-8a7d-553824965fcd None None] MFA Rules not processed for user `32da9a3fd2d44b34b27c4f166ecc7ecf`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:55:30.284507 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-f95327fd-ff5c-462e-8a7d-553824965fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:55:30.356957 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1315/2619] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:55:30 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:55:30.369163 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c371b2-89f2-455a-bdda-fbf55f88f877 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:30.371127 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-26c371b2-89f2-455a-bdda-fbf55f88f877 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:55:30.439936 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c371b2-89f2-455a-bdda-fbf55f88f877 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:30.443555 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb83a52-f5f2-4596-a212-d25173d878e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:30.444162 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb83a52-f5f2-4596-a212-d25173d878e7 service cinder] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:30.444444 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb83a52-f5f2-4596-a212-d25173d878e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:30.444714 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb83a52-f5f2-4596-a212-d25173d878e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:30.458262 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:55:30.513441 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb83a52-f5f2-4596-a212-d25173d878e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:30.515210 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb83a52-f5f2-4596-a212-d25173d878e7 service cinder] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:30.525345 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1305/2620] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:55:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:55:33.066846 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95327fd-ff5c-462e-8a7d-553824965fcd None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:33.115796 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:55:33.128973 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95327fd-ff5c-462e-8a7d-553824965fcd None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:33.134144 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccb2db6-f5ed-4c72-8b1f-8cf82d090de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:33.134768 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb2db6-f5ed-4c72-8b1f-8cf82d090de8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:33.135045 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb2db6-f5ed-4c72-8b1f-8cf82d090de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:33.135311 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb2db6-f5ed-4c72-8b1f-8cf82d090de8 admin admin] PATH_INFO: `/v3/users/ff3a246591cb4bcea9e8d8808f8a9575` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:33.136456 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccb2db6-f5ed-4c72-8b1f-8cf82d090de8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff3a246591cb4bcea9e8d8808f8a9575)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:33.138351 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccb2db6-f5ed-4c72-8b1f-8cf82d090de8 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:33.184802 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-fccb2db6-f5ed-4c72-8b1f-8cf82d090de8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff3a246591cb4bcea9e8d8808f8a9575'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:33.191910 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-fccb2db6-f5ed-4c72-8b1f-8cf82d090de8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff3a246591cb4bcea9e8d8808f8a9575'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:33.192531 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-fccb2db6-f5ed-4c72-8b1f-8cf82d090de8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff3a246591cb4bcea9e8d8808f8a9575'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:33.193159 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-fccb2db6-f5ed-4c72-8b1f-8cf82d090de8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff3a246591cb4bcea9e8d8808f8a9575'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:33.205816 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-fccb2db6-f5ed-4c72-8b1f-8cf82d090de8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff3a246591cb4bcea9e8d8808f8a9575'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:33.217691 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1316/2621] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:55:33 2022] DELETE /identity/v3/users/ff3a246591cb4bcea9e8d8808f8a9575 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:55:33.222145 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f606983-d482-4754-a471-4d8b029d2346 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:33.222671 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f606983-d482-4754-a471-4d8b029d2346 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:33.222844 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f606983-d482-4754-a471-4d8b029d2346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:33.242218 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-3f606983-d482-4754-a471-4d8b029d2346 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:55:33.244437 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-3f606983-d482-4754-a471-4d8b029d2346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:55:33.302275 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1306/2622] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:55:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:55:33.313117 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccb2db6-f5ed-4c72-8b1f-8cf82d090de8 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:33.314616 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-fccb2db6-f5ed-4c72-8b1f-8cf82d090de8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:55:33.372297 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccb2db6-f5ed-4c72-8b1f-8cf82d090de8 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:33.375962 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704fe02e-ff98-46af-b99b-bc9dec865f50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:33.376574 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704fe02e-ff98-46af-b99b-bc9dec865f50 service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:33.376858 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704fe02e-ff98-46af-b99b-bc9dec865f50 service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:33.377132 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704fe02e-ff98-46af-b99b-bc9dec865f50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:33.385925 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704fe02e-ff98-46af-b99b-bc9dec865f50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:33.387347 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704fe02e-ff98-46af-b99b-bc9dec865f50 service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:33.396692 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1317/2623] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:55:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:55:33.657278 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f606983-d482-4754-a471-4d8b029d2346 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:33.658792 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-3f606983-d482-4754-a471-4d8b029d2346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:55:33.671389 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f606983-d482-4754-a471-4d8b029d2346 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:33.675930 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4f5f7a-a29a-42c4-80b2-d42b11218f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:33.676559 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f5f7a-a29a-42c4-80b2-d42b11218f69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:33.676839 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f5f7a-a29a-42c4-80b2-d42b11218f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:33.677111 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f5f7a-a29a-42c4-80b2-d42b11218f69 admin admin] PATH_INFO: `/v3/projects/01fe93ffa5554f24bbfa220b5b0dbb40` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:33.678229 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4f5f7a-a29a-42c4-80b2-d42b11218f69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01fe93ffa5554f24bbfa220b5b0dbb40)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:33.680156 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4f5f7a-a29a-42c4-80b2-d42b11218f69 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:33.728748 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-0b4f5f7a-a29a-42c4-80b2-d42b11218f69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01fe93ffa5554f24bbfa220b5b0dbb40', 'request_id': 'req-0b4f5f7a-a29a-42c4-80b2-d42b11218f69'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:33.735110 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-0b4f5f7a-a29a-42c4-80b2-d42b11218f69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01fe93ffa5554f24bbfa220b5b0dbb40', 'request_id': 'req-0b4f5f7a-a29a-42c4-80b2-d42b11218f69'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:33.742393 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-0b4f5f7a-a29a-42c4-80b2-d42b11218f69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01fe93ffa5554f24bbfa220b5b0dbb40', 'request_id': 'req-0b4f5f7a-a29a-42c4-80b2-d42b11218f69'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:33.752728 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-0b4f5f7a-a29a-42c4-80b2-d42b11218f69 admin admin] The token cache is being invalidate because project 01fe93ffa5554f24bbfa220b5b0dbb40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90653) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:55:33.753016 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-0b4f5f7a-a29a-42c4-80b2-d42b11218f69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:33.753637 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-0b4f5f7a-a29a-42c4-80b2-d42b11218f69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:33.755088 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1307/2624] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:55:33 2022] DELETE /identity/v3/projects/01fe93ffa5554f24bbfa220b5b0dbb40 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:55:33.759268 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704fe02e-ff98-46af-b99b-bc9dec865f50 service neutron] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:33.768816 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704fe02e-ff98-46af-b99b-bc9dec865f50 service neutron] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:33.825744 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cab87e-f9a0-4981-ba18-f1e53f2b09f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:33.826377 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cab87e-f9a0-4981-ba18-f1e53f2b09f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:33.826709 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cab87e-f9a0-4981-ba18-f1e53f2b09f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:33.827089 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cab87e-f9a0-4981-ba18-f1e53f2b09f5 admin admin] PATH_INFO: `/v3/users/32da9a3fd2d44b34b27c4f166ecc7ecf` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:33.828289 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cab87e-f9a0-4981-ba18-f1e53f2b09f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32da9a3fd2d44b34b27c4f166ecc7ecf)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:33.830182 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cab87e-f9a0-4981-ba18-f1e53f2b09f5 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:33.845872 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:55:33.878016 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-56cab87e-f9a0-4981-ba18-f1e53f2b09f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32da9a3fd2d44b34b27c4f166ecc7ecf'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:33.885101 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-56cab87e-f9a0-4981-ba18-f1e53f2b09f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32da9a3fd2d44b34b27c4f166ecc7ecf'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:33.885722 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-56cab87e-f9a0-4981-ba18-f1e53f2b09f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32da9a3fd2d44b34b27c4f166ecc7ecf'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:33.886327 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-56cab87e-f9a0-4981-ba18-f1e53f2b09f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32da9a3fd2d44b34b27c4f166ecc7ecf'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:33.898133 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-56cab87e-f9a0-4981-ba18-f1e53f2b09f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32da9a3fd2d44b34b27c4f166ecc7ecf'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:33.910997 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1318/2625] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:55:33 2022] DELETE /identity/v3/users/32da9a3fd2d44b34b27c4f166ecc7ecf => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:55:34.256498 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4f5f7a-a29a-42c4-80b2-d42b11218f69 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:34.258122 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-0b4f5f7a-a29a-42c4-80b2-d42b11218f69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:55:34.272054 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4f5f7a-a29a-42c4-80b2-d42b11218f69 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:34.311028 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:55:34.336316 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bef68a9-34ff-426d-b34a-9a8669a03ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:34.337513 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bef68a9-34ff-426d-b34a-9a8669a03ad6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:34.337850 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bef68a9-34ff-426d-b34a-9a8669a03ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:34.338132 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bef68a9-34ff-426d-b34a-9a8669a03ad6 admin admin] PATH_INFO: `/v3/projects/6381f1c748d24833abc3a2cb5ba22368` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:34.339301 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bef68a9-34ff-426d-b34a-9a8669a03ad6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6381f1c748d24833abc3a2cb5ba22368)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:34.341253 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bef68a9-34ff-426d-b34a-9a8669a03ad6 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:34.388268 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-1bef68a9-34ff-426d-b34a-9a8669a03ad6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6381f1c748d24833abc3a2cb5ba22368', 'request_id': 'req-1bef68a9-34ff-426d-b34a-9a8669a03ad6'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:34.394621 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-1bef68a9-34ff-426d-b34a-9a8669a03ad6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6381f1c748d24833abc3a2cb5ba22368', 'request_id': 'req-1bef68a9-34ff-426d-b34a-9a8669a03ad6'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:34.403065 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-1bef68a9-34ff-426d-b34a-9a8669a03ad6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6381f1c748d24833abc3a2cb5ba22368', 'request_id': 'req-1bef68a9-34ff-426d-b34a-9a8669a03ad6'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:34.413811 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-1bef68a9-34ff-426d-b34a-9a8669a03ad6 admin admin] The token cache is being invalidate because project 6381f1c748d24833abc3a2cb5ba22368 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90653) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:55:34.414098 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-1bef68a9-34ff-426d-b34a-9a8669a03ad6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:34.415036 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-1bef68a9-34ff-426d-b34a-9a8669a03ad6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:55:34.416541 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1308/2626] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:55:34 2022] DELETE /identity/v3/projects/6381f1c748d24833abc3a2cb5ba22368 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 10:55:34.421466 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da42d8f-181e-43df-99b0-bafd23b93582 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:34.421921 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da42d8f-181e-43df-99b0-bafd23b93582 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:34.422017 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da42d8f-181e-43df-99b0-bafd23b93582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:34.441556 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-9da42d8f-181e-43df-99b0-bafd23b93582 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:55:34.442749 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-9da42d8f-181e-43df-99b0-bafd23b93582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:55:34.503661 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1319/2627] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:55:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 10:55:34.511973 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173fe923-6fa1-4d14-a5dc-a110b71acc5e None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:34.512257 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173fe923-6fa1-4d14-a5dc-a110b71acc5e None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:34.512529 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173fe923-6fa1-4d14-a5dc-a110b71acc5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:34.532499 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-173fe923-6fa1-4d14-a5dc-a110b71acc5e None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:55:34.533703 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-173fe923-6fa1-4d14-a5dc-a110b71acc5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:55:34.541647 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1309/2628] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:55:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:55:34.545983 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da42d8f-181e-43df-99b0-bafd23b93582 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:34.556148 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da42d8f-181e-43df-99b0-bafd23b93582 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:34.560897 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3854367e-791b-4867-98fd-e0ce2794832a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:34.561481 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3854367e-791b-4867-98fd-e0ce2794832a admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:34.561763 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3854367e-791b-4867-98fd-e0ce2794832a admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:34.562067 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3854367e-791b-4867-98fd-e0ce2794832a admin admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:34.562713 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3854367e-791b-4867-98fd-e0ce2794832a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:55:34.563013 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3854367e-791b-4867-98fd-e0ce2794832a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:34.564651 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3854367e-791b-4867-98fd-e0ce2794832a admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:34.576851 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1320/2629] 10.222.0.15 () {58 vars in 1153 bytes} [Sun Sep 11 10:55:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:55:34.580990 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173fe923-6fa1-4d14-a5dc-a110b71acc5e None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:34.590477 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173fe923-6fa1-4d14-a5dc-a110b71acc5e None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:34.595007 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19fac4f-10cc-4ef0-a684-19250d5d2429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:34.595614 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19fac4f-10cc-4ef0-a684-19250d5d2429 admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:34.595890 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19fac4f-10cc-4ef0-a684-19250d5d2429 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:34.596157 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19fac4f-10cc-4ef0-a684-19250d5d2429 admin admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:34.596797 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19fac4f-10cc-4ef0-a684-19250d5d2429 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:34.598663 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19fac4f-10cc-4ef0-a684-19250d5d2429 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:34.626558 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1310/2630] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:55:34 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:55:34.630607 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3854367e-791b-4867-98fd-e0ce2794832a admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:34.640519 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3854367e-791b-4867-98fd-e0ce2794832a admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:34.697438 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977ed23f-4a0e-4631-b342-8d22895bbc50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:34.698280 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ed23f-4a0e-4631-b342-8d22895bbc50 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:34.698569 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ed23f-4a0e-4631-b342-8d22895bbc50 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:34.698843 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ed23f-4a0e-4631-b342-8d22895bbc50 admin admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:34.699498 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977ed23f-4a0e-4631-b342-8d22895bbc50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:34.701414 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977ed23f-4a0e-4631-b342-8d22895bbc50 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:34.723372 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1321/2631] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:55:34 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:55:34.729783 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19fac4f-10cc-4ef0-a684-19250d5d2429 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:34.740584 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19fac4f-10cc-4ef0-a684-19250d5d2429 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:34.745380 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0799a2b-4e2e-4676-a38f-48517abb185d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:34.745984 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0799a2b-4e2e-4676-a38f-48517abb185d admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:34.746270 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0799a2b-4e2e-4676-a38f-48517abb185d admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:34.746542 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0799a2b-4e2e-4676-a38f-48517abb185d admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:34.747179 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0799a2b-4e2e-4676-a38f-48517abb185d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:34.748795 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0799a2b-4e2e-4676-a38f-48517abb185d admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:34.759494 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1311/2632] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:55:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:55:34.763565 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977ed23f-4a0e-4631-b342-8d22895bbc50 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:34.773527 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977ed23f-4a0e-4631-b342-8d22895bbc50 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:34.779059 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8990376e-221b-4824-a98a-e3bd64b03d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:34.779695 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8990376e-221b-4824-a98a-e3bd64b03d99 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:34.779993 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8990376e-221b-4824-a98a-e3bd64b03d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:34.780251 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8990376e-221b-4824-a98a-e3bd64b03d99 admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:34.780891 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8990376e-221b-4824-a98a-e3bd64b03d99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:34.782480 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8990376e-221b-4824-a98a-e3bd64b03d99 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:34.785085 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:55:34.797215 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1322/2633] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:55:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:55:34.801266 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0799a2b-4e2e-4676-a38f-48517abb185d admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:34.811515 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0799a2b-4e2e-4676-a38f-48517abb185d admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:34.816227 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987e9b3a-9833-43b5-95b3-dd915ae212d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:34.816895 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987e9b3a-9833-43b5-95b3-dd915ae212d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:34.817171 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987e9b3a-9833-43b5-95b3-dd915ae212d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:34.817438 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987e9b3a-9833-43b5-95b3-dd915ae212d1 admin admin] PATH_INFO: `/v3/projects/80459f77fe204aefb516e8389f1458fc/users/fc2f971c8c4d47f59c79495289e59112/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:34.832554 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987e9b3a-9833-43b5-95b3-dd915ae212d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80459f77fe204aefb516e8389f1458fc, user_id=fc2f971c8c4d47f59c79495289e59112, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:34.835190 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987e9b3a-9833-43b5-95b3-dd915ae212d1 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:34.847913 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1312/2634] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:55:34 2022] PUT /identity/v3/projects/80459f77fe204aefb516e8389f1458fc/users/fc2f971c8c4d47f59c79495289e59112/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:55:34.853364 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3798562c-ea50-4e4c-ba99-d5c77e3a040b None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:34.853647 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3798562c-ea50-4e4c-ba99-d5c77e3a040b None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:34.853917 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3798562c-ea50-4e4c-ba99-d5c77e3a040b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:34.873146 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-3798562c-ea50-4e4c-ba99-d5c77e3a040b None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:55:34.874337 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-3798562c-ea50-4e4c-ba99-d5c77e3a040b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:55:34.935446 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1323/2635] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:55:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:55:34.944958 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987e9b3a-9833-43b5-95b3-dd915ae212d1 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:34.946813 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-987e9b3a-9833-43b5-95b3-dd915ae212d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:55:35.004368 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987e9b3a-9833-43b5-95b3-dd915ae212d1 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:35.007998 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17735644-f086-49de-8e59-1e0af43c3aa2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:35.008831 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17735644-f086-49de-8e59-1e0af43c3aa2 service neutron] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:35.009257 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17735644-f086-49de-8e59-1e0af43c3aa2 service neutron] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:35.009539 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17735644-f086-49de-8e59-1e0af43c3aa2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:35.018416 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17735644-f086-49de-8e59-1e0af43c3aa2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:35.019867 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17735644-f086-49de-8e59-1e0af43c3aa2 service neutron] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:35.029339 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1313/2636] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:55:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:55:35.715191 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788a465b-feb2-4d4d-a329-3c50a2c4d035 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:35.715743 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788a465b-feb2-4d4d-a329-3c50a2c4d035 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:35.716228 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788a465b-feb2-4d4d-a329-3c50a2c4d035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:35.764012 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-788a465b-feb2-4d4d-a329-3c50a2c4d035 None None] MFA Rules not processed for user `fc2f971c8c4d47f59c79495289e59112`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:55:35.766140 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-788a465b-feb2-4d4d-a329-3c50a2c4d035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:55:35.849955 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1324/2637] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:55:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 10:55:35.860807 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17735644-f086-49de-8e59-1e0af43c3aa2 service neutron] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:55:35.862197 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-17735644-f086-49de-8e59-1e0af43c3aa2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:55:35.873136 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:55:35.926806 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17735644-f086-49de-8e59-1e0af43c3aa2 service neutron] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:55:35.930915 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c282cde-9b14-4859-adf4-536c8c691b71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:55:35.931667 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c282cde-9b14-4859-adf4-536c8c691b71 service cinder] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:35.931945 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c282cde-9b14-4859-adf4-536c8c691b71 service cinder] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:35.932214 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c282cde-9b14-4859-adf4-536c8c691b71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:35.940490 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c282cde-9b14-4859-adf4-536c8c691b71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:55:35.941886 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c282cde-9b14-4859-adf4-536c8c691b71 service cinder] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:55:35.951562 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1314/2638] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:55:35 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:55:37.344980 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a2fa8e-6a0d-4953-8171-11281de5af8c None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:55:37.345457 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a2fa8e-6a0d-4953-8171-11281de5af8c None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:55:37.345557 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a2fa8e-6a0d-4953-8171-11281de5af8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:55:37.364320 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-b5a2fa8e-6a0d-4953-8171-11281de5af8c None None] MFA Rules not processed for user `fc2f971c8c4d47f59c79495289e59112`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:55:37.365501 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-b5a2fa8e-6a0d-4953-8171-11281de5af8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:55:37.372223 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1325/2639] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:55:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:55:45.947624 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=23 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:55:47.359566 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=63 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:56:02.952609 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c282cde-9b14-4859-adf4-536c8c691b71 service cinder] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:56:02.961905 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c282cde-9b14-4859-adf4-536c8c691b71 service cinder] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:56:02.966174 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86782772-c7f4-49ec-8eb5-69a466e4992f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:56:02.966763 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86782772-c7f4-49ec-8eb5-69a466e4992f service neutron] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:56:02.967067 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86782772-c7f4-49ec-8eb5-69a466e4992f service neutron] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:56:02.967338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86782772-c7f4-49ec-8eb5-69a466e4992f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:56:02.969243 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-86782772-c7f4-49ec-8eb5-69a466e4992f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:56:03.029505 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86782772-c7f4-49ec-8eb5-69a466e4992f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:56:03.030924 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86782772-c7f4-49ec-8eb5-69a466e4992f service neutron] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:56:03.045143 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1315/2640] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:56:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:56:09.131639 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a2fa8e-6a0d-4953-8171-11281de5af8c None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:56:09.133071 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-b5a2fa8e-6a0d-4953-8171-11281de5af8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:56:09.145792 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a2fa8e-6a0d-4953-8171-11281de5af8c None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:56:09.149347 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee92b5c-1021-4c19-82fb-8c4ce68507b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:56:09.150025 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee92b5c-1021-4c19-82fb-8c4ce68507b7 service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:56:09.150325 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee92b5c-1021-4c19-82fb-8c4ce68507b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:56:09.150596 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee92b5c-1021-4c19-82fb-8c4ce68507b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:56:09.152349 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-2ee92b5c-1021-4c19-82fb-8c4ce68507b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:56:09.164802 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee92b5c-1021-4c19-82fb-8c4ce68507b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:56:09.166224 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee92b5c-1021-4c19-82fb-8c4ce68507b7 service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:56:09.176423 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1326/2641] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:56:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:56:13.040301 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=28 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:56:15.350950 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86782772-c7f4-49ec-8eb5-69a466e4992f service neutron] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:56:15.352686 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-86782772-c7f4-49ec-8eb5-69a466e4992f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:56:15.368316 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86782772-c7f4-49ec-8eb5-69a466e4992f service neutron] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:56:15.372777 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6203c9e6-4c62-4a21-be09-cb5c1c05e7cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:56:15.374130 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6203c9e6-4c62-4a21-be09-cb5c1c05e7cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:56:15.374494 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6203c9e6-4c62-4a21-be09-cb5c1c05e7cc admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:56:15.374767 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6203c9e6-4c62-4a21-be09-cb5c1c05e7cc admin admin] PATH_INFO: `/v3/users/e839e78f400a46eb916b51c9e257bdf1` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:56:15.375895 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6203c9e6-4c62-4a21-be09-cb5c1c05e7cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=e839e78f400a46eb916b51c9e257bdf1)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:56:15.377773 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6203c9e6-4c62-4a21-be09-cb5c1c05e7cc admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:56:15.424668 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-6203c9e6-4c62-4a21-be09-cb5c1c05e7cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e839e78f400a46eb916b51c9e257bdf1'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:56:15.431703 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-6203c9e6-4c62-4a21-be09-cb5c1c05e7cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e839e78f400a46eb916b51c9e257bdf1'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:56:15.432415 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-6203c9e6-4c62-4a21-be09-cb5c1c05e7cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e839e78f400a46eb916b51c9e257bdf1'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:56:15.433020 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-6203c9e6-4c62-4a21-be09-cb5c1c05e7cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e839e78f400a46eb916b51c9e257bdf1'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:56:15.440630 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-6203c9e6-4c62-4a21-be09-cb5c1c05e7cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e839e78f400a46eb916b51c9e257bdf1'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:56:15.458681 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1316/2642] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:56:15 2022] DELETE /identity/v3/users/e839e78f400a46eb916b51c9e257bdf1 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:56:15.632208 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee92b5c-1021-4c19-82fb-8c4ce68507b7 service neutron] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:56:15.642203 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee92b5c-1021-4c19-82fb-8c4ce68507b7 service neutron] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:56:15.700977 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd71838-fd3f-4d19-9421-3df1fc0b7890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:56:15.701649 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd71838-fd3f-4d19-9421-3df1fc0b7890 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:56:15.701924 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd71838-fd3f-4d19-9421-3df1fc0b7890 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:56:15.702196 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd71838-fd3f-4d19-9421-3df1fc0b7890 admin admin] PATH_INFO: `/v3/projects/587882642889489587749d5fb846acc6` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:56:15.703307 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd71838-fd3f-4d19-9421-3df1fc0b7890 admin admin] RBAC: Authorizing `identity:delete_project(project_id=587882642889489587749d5fb846acc6)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:56:15.705238 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd71838-fd3f-4d19-9421-3df1fc0b7890 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:56:15.754803 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-2dd71838-fd3f-4d19-9421-3df1fc0b7890 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '587882642889489587749d5fb846acc6', 'request_id': 'req-2dd71838-fd3f-4d19-9421-3df1fc0b7890'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:56:15.761125 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-2dd71838-fd3f-4d19-9421-3df1fc0b7890 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '587882642889489587749d5fb846acc6', 'request_id': 'req-2dd71838-fd3f-4d19-9421-3df1fc0b7890'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:56:15.766782 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-2dd71838-fd3f-4d19-9421-3df1fc0b7890 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '587882642889489587749d5fb846acc6', 'request_id': 'req-2dd71838-fd3f-4d19-9421-3df1fc0b7890'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:56:15.778689 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-2dd71838-fd3f-4d19-9421-3df1fc0b7890 admin admin] The token cache is being invalidate because project 587882642889489587749d5fb846acc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:56:15.778983 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-2dd71838-fd3f-4d19-9421-3df1fc0b7890 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:56:15.779780 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-2dd71838-fd3f-4d19-9421-3df1fc0b7890 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:56:15.781458 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1327/2643] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:56:15 2022] DELETE /identity/v3/projects/587882642889489587749d5fb846acc6 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:56:15.785380 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6203c9e6-4c62-4a21-be09-cb5c1c05e7cc admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:56:15.786773 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-6203c9e6-4c62-4a21-be09-cb5c1c05e7cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:56:15.800406 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6203c9e6-4c62-4a21-be09-cb5c1c05e7cc admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:56:15.856169 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaef4bd-dfb6-4ffd-a075-69f0ca4992cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:56:15.856807 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaef4bd-dfb6-4ffd-a075-69f0ca4992cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:56:15.857085 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaef4bd-dfb6-4ffd-a075-69f0ca4992cb admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:56:15.857868 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaef4bd-dfb6-4ffd-a075-69f0ca4992cb admin admin] PATH_INFO: `/v3/users/2eb2819aee9646d692373511ed762e8a` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:56:15.860319 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaef4bd-dfb6-4ffd-a075-69f0ca4992cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=2eb2819aee9646d692373511ed762e8a)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:56:15.861194 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaef4bd-dfb6-4ffd-a075-69f0ca4992cb admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:56:15.910156 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-faaef4bd-dfb6-4ffd-a075-69f0ca4992cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2eb2819aee9646d692373511ed762e8a'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:56:15.918182 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-faaef4bd-dfb6-4ffd-a075-69f0ca4992cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2eb2819aee9646d692373511ed762e8a'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:56:15.918886 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-faaef4bd-dfb6-4ffd-a075-69f0ca4992cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2eb2819aee9646d692373511ed762e8a'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:56:15.919592 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-faaef4bd-dfb6-4ffd-a075-69f0ca4992cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2eb2819aee9646d692373511ed762e8a'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:56:15.927737 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-faaef4bd-dfb6-4ffd-a075-69f0ca4992cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2eb2819aee9646d692373511ed762e8a'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:56:15.936435 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:56:15.950659 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1317/2644] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:56:15 2022] DELETE /identity/v3/users/2eb2819aee9646d692373511ed762e8a => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:56:16.122650 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd71838-fd3f-4d19-9421-3df1fc0b7890 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:56:16.125856 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-2dd71838-fd3f-4d19-9421-3df1fc0b7890 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:56:16.140573 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd71838-fd3f-4d19-9421-3df1fc0b7890 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:56:16.205955 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05c3c7d-454a-49e9-bf13-63ca82bc4cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:56:16.206497 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05c3c7d-454a-49e9-bf13-63ca82bc4cff admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:56:16.206778 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05c3c7d-454a-49e9-bf13-63ca82bc4cff admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:56:16.207051 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05c3c7d-454a-49e9-bf13-63ca82bc4cff admin admin] PATH_INFO: `/v3/projects/642eea5bb9754514a78a7d16660332ee` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:56:16.208303 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05c3c7d-454a-49e9-bf13-63ca82bc4cff admin admin] RBAC: Authorizing `identity:delete_project(project_id=642eea5bb9754514a78a7d16660332ee)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:56:16.210213 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05c3c7d-454a-49e9-bf13-63ca82bc4cff admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:56:16.234994 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:56:16.265867 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-b05c3c7d-454a-49e9-bf13-63ca82bc4cff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '642eea5bb9754514a78a7d16660332ee', 'request_id': 'req-b05c3c7d-454a-49e9-bf13-63ca82bc4cff'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:56:16.272583 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-b05c3c7d-454a-49e9-bf13-63ca82bc4cff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '642eea5bb9754514a78a7d16660332ee', 'request_id': 'req-b05c3c7d-454a-49e9-bf13-63ca82bc4cff'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:56:16.278832 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-b05c3c7d-454a-49e9-bf13-63ca82bc4cff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '642eea5bb9754514a78a7d16660332ee', 'request_id': 'req-b05c3c7d-454a-49e9-bf13-63ca82bc4cff'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:56:16.302813 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-b05c3c7d-454a-49e9-bf13-63ca82bc4cff admin admin] The token cache is being invalidate because project 642eea5bb9754514a78a7d16660332ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:56:16.303104 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-b05c3c7d-454a-49e9-bf13-63ca82bc4cff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:56:16.303751 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-b05c3c7d-454a-49e9-bf13-63ca82bc4cff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:56:16.305274 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1328/2645] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:56:16 2022] DELETE /identity/v3/projects/642eea5bb9754514a78a7d16660332ee => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:56:25.949332 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=3,keystone:DELETE=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:56:26.296795 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:56:37.404656 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaef4bd-dfb6-4ffd-a075-69f0ca4992cb admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:56:37.406066 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-faaef4bd-dfb6-4ffd-a075-69f0ca4992cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:56:37.463571 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaef4bd-dfb6-4ffd-a075-69f0ca4992cb admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:56:37.511866 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bacdec-d4fa-4341-989f-8a84826f1ad5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:56:37.512468 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bacdec-d4fa-4341-989f-8a84826f1ad5 service neutron] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:56:37.512745 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bacdec-d4fa-4341-989f-8a84826f1ad5 service neutron] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:56:37.513013 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bacdec-d4fa-4341-989f-8a84826f1ad5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:56:37.514793 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-71bacdec-d4fa-4341-989f-8a84826f1ad5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:56:37.596352 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bacdec-d4fa-4341-989f-8a84826f1ad5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:56:37.597676 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bacdec-d4fa-4341-989f-8a84826f1ad5 service neutron] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:56:37.618695 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1318/2646] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:56:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:56:38.804343 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05c3c7d-454a-49e9-bf13-63ca82bc4cff admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:56:38.805938 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-b05c3c7d-454a-49e9-bf13-63ca82bc4cff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:56:38.820432 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05c3c7d-454a-49e9-bf13-63ca82bc4cff admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:56:38.824211 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b35a4c7-830e-4e3d-a952-09471cb0fe95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:56:38.824986 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b35a4c7-830e-4e3d-a952-09471cb0fe95 service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:56:38.825270 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b35a4c7-830e-4e3d-a952-09471cb0fe95 service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:56:38.825539 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b35a4c7-830e-4e3d-a952-09471cb0fe95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:56:38.827266 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-2b35a4c7-830e-4e3d-a952-09471cb0fe95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:56:38.841855 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b35a4c7-830e-4e3d-a952-09471cb0fe95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:56:38.843461 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b35a4c7-830e-4e3d-a952-09471cb0fe95 service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:56:38.854770 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1329/2647] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:56:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:56:45.249982 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a05f4c-fdde-4849-a83b-88c545b9b8fc None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:56:45.250464 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a05f4c-fdde-4849-a83b-88c545b9b8fc None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:56:45.250596 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a05f4c-fdde-4849-a83b-88c545b9b8fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:56:45.273843 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-95a05f4c-fdde-4849-a83b-88c545b9b8fc None None] MFA Rules not processed for user `bb44e7aef0c7436d93a7233307b5f095`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:56:45.275034 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-95a05f4c-fdde-4849-a83b-88c545b9b8fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:56:45.308998 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:56:45.334105 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1319/2648] 10.222.0.15 () {58 vars in 1009 bytes} [Sun Sep 11 10:56:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:56:45.380250 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b35a4c7-830e-4e3d-a952-09471cb0fe95 service neutron] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:56:45.381652 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-2b35a4c7-830e-4e3d-a952-09471cb0fe95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:56:45.451412 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b35a4c7-830e-4e3d-a952-09471cb0fe95 service neutron] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:56:45.456574 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15489c66-8bd8-495f-8069-edd02746dfba service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7ca566e66a45b29e3d02f47bf84981', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:56:45.457175 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15489c66-8bd8-495f-8069-edd02746dfba service swift] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:56:45.457452 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15489c66-8bd8-495f-8069-edd02746dfba service swift] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:56:45.457718 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15489c66-8bd8-495f-8069-edd02746dfba service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:56:45.465495 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15489c66-8bd8-495f-8069-edd02746dfba service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:56:45.467055 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15489c66-8bd8-495f-8069-edd02746dfba service swift] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:56:45.475156 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1330/2649] 10.222.0.15 () {60 vars in 1384 bytes} [Sun Sep 11 10:56:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 10:56:47.156567 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a05f4c-fdde-4849-a83b-88c545b9b8fc None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:56:47.172806 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a05f4c-fdde-4849-a83b-88c545b9b8fc None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:56:47.177371 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9449ee12-ce68-4f0c-84c0-374df3bc92da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:56:47.178079 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9449ee12-ce68-4f0c-84c0-374df3bc92da service neutron] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:56:47.178364 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9449ee12-ce68-4f0c-84c0-374df3bc92da service neutron] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:56:47.178900 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9449ee12-ce68-4f0c-84c0-374df3bc92da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:56:47.181403 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-9449ee12-ce68-4f0c-84c0-374df3bc92da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:56:47.317974 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9449ee12-ce68-4f0c-84c0-374df3bc92da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:56:47.319061 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9449ee12-ce68-4f0c-84c0-374df3bc92da service neutron] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:56:47.337731 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1320/2650] 10.222.0.15 () {60 vars in 1396 bytes} [Sun Sep 11 10:56:47 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3266 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:56:55.472140 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=38 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:56:57.328895 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=45 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:56:58.701178 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15489c66-8bd8-495f-8069-edd02746dfba service swift] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:56:58.702637 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-15489c66-8bd8-495f-8069-edd02746dfba service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:56:58.765653 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15489c66-8bd8-495f-8069-edd02746dfba service swift] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:56:58.769446 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bf031d-14c2-4df5-8183-1e352f056dff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:56:58.770049 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bf031d-14c2-4df5-8183-1e352f056dff service cinder] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:56:58.770329 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bf031d-14c2-4df5-8183-1e352f056dff service cinder] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:56:58.770599 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bf031d-14c2-4df5-8183-1e352f056dff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:56:58.772339 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-71bf031d-14c2-4df5-8183-1e352f056dff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:56:58.832656 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bf031d-14c2-4df5-8183-1e352f056dff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:56:58.834024 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bf031d-14c2-4df5-8183-1e352f056dff service cinder] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:56:58.843592 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1331/2651] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:56:58 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:57:04.292725 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9449ee12-ce68-4f0c-84c0-374df3bc92da service neutron] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:57:04.301746 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9449ee12-ce68-4f0c-84c0-374df3bc92da service neutron] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:57:04.305311 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a19a896-6035-49fe-9ce9-4fb740f76c38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:57:04.305947 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19a896-6035-49fe-9ce9-4fb740f76c38 service neutron] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:57:04.306278 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19a896-6035-49fe-9ce9-4fb740f76c38 service neutron] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:57:04.306546 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19a896-6035-49fe-9ce9-4fb740f76c38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:57:04.308291 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-5a19a896-6035-49fe-9ce9-4fb740f76c38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:57:04.318517 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a19a896-6035-49fe-9ce9-4fb740f76c38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:57:04.320058 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a19a896-6035-49fe-9ce9-4fb740f76c38 service neutron] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:57:04.328314 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1321/2652] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:57:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:57:08.839666 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=48 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:57:11.757617 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6093fd-b509-40f5-8f6e-71ff73c7229c None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:57:11.758222 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6093fd-b509-40f5-8f6e-71ff73c7229c None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:57:11.758222 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6093fd-b509-40f5-8f6e-71ff73c7229c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:57:11.776777 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-da6093fd-b509-40f5-8f6e-71ff73c7229c None None] MFA Rules not processed for user `fc2f971c8c4d47f59c79495289e59112`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:57:11.777968 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-da6093fd-b509-40f5-8f6e-71ff73c7229c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:57:11.836106 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1332/2653] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:57:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:57:12.078517 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a19a896-6035-49fe-9ce9-4fb740f76c38 service neutron] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:57:12.088174 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a19a896-6035-49fe-9ce9-4fb740f76c38 service neutron] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:57:12.091840 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c610910-ff34-4376-b90f-91a74f5978a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:57:12.092431 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c610910-ff34-4376-b90f-91a74f5978a9 service neutron] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:57:12.092711 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c610910-ff34-4376-b90f-91a74f5978a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:57:12.092980 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c610910-ff34-4376-b90f-91a74f5978a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:57:12.094667 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-0c610910-ff34-4376-b90f-91a74f5978a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:57:12.104859 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c610910-ff34-4376-b90f-91a74f5978a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:57:12.106259 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c610910-ff34-4376-b90f-91a74f5978a9 service neutron] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:57:12.114513 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1322/2654] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:57:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:57:15.207217 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69f30a0-c383-4ab2-b82e-a0c2d51f04d7 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:57:15.207853 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69f30a0-c383-4ab2-b82e-a0c2d51f04d7 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:57:15.207853 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69f30a0-c383-4ab2-b82e-a0c2d51f04d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:57:15.227447 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-a69f30a0-c383-4ab2-b82e-a0c2d51f04d7 None None] MFA Rules not processed for user `e1b0815853b947bc8f50d91c1ce0ab5a`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:57:15.228681 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-a69f30a0-c383-4ab2-b82e-a0c2d51f04d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:57:15.235938 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1333/2655] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:57:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:57:16.239085 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e554512-35c5-4f8c-bb27-c6e6b8ee95b9 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:57:16.239598 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e554512-35c5-4f8c-bb27-c6e6b8ee95b9 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:57:16.239704 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e554512-35c5-4f8c-bb27-c6e6b8ee95b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:57:16.290665 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-0e554512-35c5-4f8c-bb27-c6e6b8ee95b9 None None] MFA Rules not processed for user `bb44e7aef0c7436d93a7233307b5f095`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:57:16.291916 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-0e554512-35c5-4f8c-bb27-c6e6b8ee95b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:57:16.314253 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1323/2656] 10.222.0.15 () {58 vars in 1009 bytes} [Sun Sep 11 10:57:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:57:16.326493 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69f30a0-c383-4ab2-b82e-a0c2d51f04d7 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:57:16.349674 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69f30a0-c383-4ab2-b82e-a0c2d51f04d7 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:57:16.356975 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9bec6e-ca03-4f0a-83ae-2a8f2ef75485 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7ca566e66a45b29e3d02f47bf84981', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:57:16.357918 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9bec6e-ca03-4f0a-83ae-2a8f2ef75485 service swift] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:57:16.358374 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9bec6e-ca03-4f0a-83ae-2a8f2ef75485 service swift] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:57:16.359119 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9bec6e-ca03-4f0a-83ae-2a8f2ef75485 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:57:16.371445 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9bec6e-ca03-4f0a-83ae-2a8f2ef75485 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:57:16.373053 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9bec6e-ca03-4f0a-83ae-2a8f2ef75485 service swift] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:57:16.382825 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1334/2657] 10.222.0.15 () {60 vars in 1384 bytes} [Sun Sep 11 10:57:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 10:57:26.307345 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=30 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:57:26.380246 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=38 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:57:39.239150 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e554512-35c5-4f8c-bb27-c6e6b8ee95b9 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:57:39.240572 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-0e554512-35c5-4f8c-bb27-c6e6b8ee95b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:57:39.256018 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e554512-35c5-4f8c-bb27-c6e6b8ee95b9 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:57:39.262373 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f5d082-4145-4e99-ac0d-c4e14f9c1567 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bd5356829b6454993bab9bad2f79fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:57:39.262993 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f5d082-4145-4e99-ac0d-c4e14f9c1567 service nova] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:57:39.263302 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f5d082-4145-4e99-ac0d-c4e14f9c1567 service nova] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:57:39.263623 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f5d082-4145-4e99-ac0d-c4e14f9c1567 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:57:39.266026 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-c7f5d082-4145-4e99-ac0d-c4e14f9c1567 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:57:39.280338 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f5d082-4145-4e99-ac0d-c4e14f9c1567 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:57:39.281950 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f5d082-4145-4e99-ac0d-c4e14f9c1567 service nova] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:57:39.293533 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1324/2658] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:57:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:57:40.948494 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6332bd1d-8f4c-4f98-9678-74f28f845e72 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:57:40.949040 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6332bd1d-8f4c-4f98-9678-74f28f845e72 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:57:40.949654 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6332bd1d-8f4c-4f98-9678-74f28f845e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:57:40.971553 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-6332bd1d-8f4c-4f98-9678-74f28f845e72 None None] MFA Rules not processed for user `bb44e7aef0c7436d93a7233307b5f095`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:57:40.973181 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-6332bd1d-8f4c-4f98-9678-74f28f845e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:57:40.981209 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1335/2659] 10.222.0.15 () {58 vars in 967 bytes} [Sun Sep 11 10:57:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:57:40.994810 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f5d082-4145-4e99-ac0d-c4e14f9c1567 service nova] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:57:41.006845 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f5d082-4145-4e99-ac0d-c4e14f9c1567 service nova] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:57:41.012379 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1802e316-9b58-4939-b5e9-bfd333a0b509 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7ca566e66a45b29e3d02f47bf84981', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:57:41.013298 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1802e316-9b58-4939-b5e9-bfd333a0b509 service swift] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:57:41.013778 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1802e316-9b58-4939-b5e9-bfd333a0b509 service swift] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:57:41.014330 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1802e316-9b58-4939-b5e9-bfd333a0b509 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:57:41.024591 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1802e316-9b58-4939-b5e9-bfd333a0b509 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:57:41.026287 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1802e316-9b58-4939-b5e9-bfd333a0b509 service swift] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:57:41.034825 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1325/2660] 10.222.0.15 () {60 vars in 1384 bytes} [Sun Sep 11 10:57:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 10:57:50.965674 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=3 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:57:51.031433 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=12 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:57:52.228196 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6332bd1d-8f4c-4f98-9678-74f28f845e72 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:57:52.241417 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6332bd1d-8f4c-4f98-9678-74f28f845e72 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:57:52.246837 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcc5f1e-dd29-47bf-9b64-1f9f77683c48 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bd5356829b6454993bab9bad2f79fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:57:52.247495 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcc5f1e-dd29-47bf-9b64-1f9f77683c48 service nova] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:57:52.247778 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcc5f1e-dd29-47bf-9b64-1f9f77683c48 service nova] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:57:52.248053 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcc5f1e-dd29-47bf-9b64-1f9f77683c48 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:57:52.249743 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-dfcc5f1e-dd29-47bf-9b64-1f9f77683c48 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:57:52.334995 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcc5f1e-dd29-47bf-9b64-1f9f77683c48 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:57:52.336633 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcc5f1e-dd29-47bf-9b64-1f9f77683c48 service nova] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:57:52.355518 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1336/2661] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:57:52 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:57:59.895959 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1802e316-9b58-4939-b5e9-bfd333a0b509 service swift] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:57:59.897700 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-1802e316-9b58-4939-b5e9-bfd333a0b509 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:57:59.910150 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1802e316-9b58-4939-b5e9-bfd333a0b509 service swift] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:57:59.913639 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2268cd92-f104-4eec-9632-5bd84f8efc12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:57:59.914225 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2268cd92-f104-4eec-9632-5bd84f8efc12 service neutron] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:57:59.914500 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2268cd92-f104-4eec-9632-5bd84f8efc12 service neutron] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:57:59.914766 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2268cd92-f104-4eec-9632-5bd84f8efc12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:57:59.916438 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-2268cd92-f104-4eec-9632-5bd84f8efc12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:57:59.978896 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2268cd92-f104-4eec-9632-5bd84f8efc12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:57:59.980345 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2268cd92-f104-4eec-9632-5bd84f8efc12 service neutron] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:57:59.990753 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1326/2662] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:57:59 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:58:02.352847 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=41 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:58:09.985703 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=30 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:58:44.228981 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcc5f1e-dd29-47bf-9b64-1f9f77683c48 service nova] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:58:44.230400 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-dfcc5f1e-dd29-47bf-9b64-1f9f77683c48 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:58:44.317273 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcc5f1e-dd29-47bf-9b64-1f9f77683c48 service nova] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:58:44.317856 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2268cd92-f104-4eec-9632-5bd84f8efc12 service neutron] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:58:44.320183 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-2268cd92-f104-4eec-9632-5bd84f8efc12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:58:44.327400 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cee7615-26d3-4587-bb08-a1d3db66df0f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08477b4c88f04c9e97391aba401879e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:58:44.328292 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cee7615-26d3-4587-bb08-a1d3db66df0f service placement] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:44.328292 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cee7615-26d3-4587-bb08-a1d3db66df0f service placement] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:44.328593 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cee7615-26d3-4587-bb08-a1d3db66df0f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:44.330385 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-5cee7615-26d3-4587-bb08-a1d3db66df0f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:58:44.348921 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cee7615-26d3-4587-bb08-a1d3db66df0f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:58:44.350600 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cee7615-26d3-4587-bb08-a1d3db66df0f service placement] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:58:44.361425 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1337/2663] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:58:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:58:44.401213 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cee7615-26d3-4587-bb08-a1d3db66df0f service placement] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:58:44.405402 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2268cd92-f104-4eec-9632-5bd84f8efc12 service neutron] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:58:44.411001 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e23413-951d-4689-9f74-c76d0b815b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:58:44.411666 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e23413-951d-4689-9f74-c76d0b815b46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:44.412047 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e23413-951d-4689-9f74-c76d0b815b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:44.412325 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e23413-951d-4689-9f74-c76d0b815b46 admin admin] PATH_INFO: `/v3/users/fc2f971c8c4d47f59c79495289e59112` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:44.413203 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cee7615-26d3-4587-bb08-a1d3db66df0f service placement] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:58:44.413467 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e23413-951d-4689-9f74-c76d0b815b46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc2f971c8c4d47f59c79495289e59112)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:58:44.415653 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e23413-951d-4689-9f74-c76d0b815b46 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:58:44.416818 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be640aba-d25a-46b3-a089-9efbfa5746f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:58:44.417416 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be640aba-d25a-46b3-a089-9efbfa5746f5 service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:44.417689 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be640aba-d25a-46b3-a089-9efbfa5746f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:44.417983 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be640aba-d25a-46b3-a089-9efbfa5746f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:44.419731 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-be640aba-d25a-46b3-a089-9efbfa5746f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:58:44.432984 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be640aba-d25a-46b3-a089-9efbfa5746f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:58:44.434346 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be640aba-d25a-46b3-a089-9efbfa5746f5 service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:58:44.445098 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1338/2664] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:58:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:58:44.464357 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-b6e23413-951d-4689-9f74-c76d0b815b46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc2f971c8c4d47f59c79495289e59112'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:44.471878 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-b6e23413-951d-4689-9f74-c76d0b815b46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc2f971c8c4d47f59c79495289e59112'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:44.472761 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-b6e23413-951d-4689-9f74-c76d0b815b46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc2f971c8c4d47f59c79495289e59112'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:44.473949 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-b6e23413-951d-4689-9f74-c76d0b815b46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc2f971c8c4d47f59c79495289e59112'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:44.481152 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-b6e23413-951d-4689-9f74-c76d0b815b46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc2f971c8c4d47f59c79495289e59112'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:44.499308 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1327/2665] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:58:44 2022] DELETE /identity/v3/users/fc2f971c8c4d47f59c79495289e59112 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 10:58:44.669966 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be640aba-d25a-46b3-a089-9efbfa5746f5 service neutron] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:58:44.680375 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be640aba-d25a-46b3-a089-9efbfa5746f5 service neutron] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:58:44.742216 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977ffe7a-c5f7-48d3-b2d2-93b117297ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:58:44.742887 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ffe7a-c5f7-48d3-b2d2-93b117297ad1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:44.743160 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ffe7a-c5f7-48d3-b2d2-93b117297ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:44.743448 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ffe7a-c5f7-48d3-b2d2-93b117297ad1 admin admin] PATH_INFO: `/v3/projects/80459f77fe204aefb516e8389f1458fc` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:44.744585 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977ffe7a-c5f7-48d3-b2d2-93b117297ad1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80459f77fe204aefb516e8389f1458fc)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:58:44.746527 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977ffe7a-c5f7-48d3-b2d2-93b117297ad1 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:58:44.797860 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-977ffe7a-c5f7-48d3-b2d2-93b117297ad1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80459f77fe204aefb516e8389f1458fc', 'request_id': 'req-977ffe7a-c5f7-48d3-b2d2-93b117297ad1'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:44.804753 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-977ffe7a-c5f7-48d3-b2d2-93b117297ad1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80459f77fe204aefb516e8389f1458fc', 'request_id': 'req-977ffe7a-c5f7-48d3-b2d2-93b117297ad1'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:44.810492 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-977ffe7a-c5f7-48d3-b2d2-93b117297ad1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80459f77fe204aefb516e8389f1458fc', 'request_id': 'req-977ffe7a-c5f7-48d3-b2d2-93b117297ad1'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:44.823967 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-977ffe7a-c5f7-48d3-b2d2-93b117297ad1 admin admin] The token cache is being invalidate because project 80459f77fe204aefb516e8389f1458fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:58:44.824253 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-977ffe7a-c5f7-48d3-b2d2-93b117297ad1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:44.824970 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-977ffe7a-c5f7-48d3-b2d2-93b117297ad1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:44.826505 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1339/2666] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:58:44 2022] DELETE /identity/v3/projects/80459f77fe204aefb516e8389f1458fc => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:58:47.611896 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e23413-951d-4689-9f74-c76d0b815b46 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:58:47.613803 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-b6e23413-951d-4689-9f74-c76d0b815b46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:58:47.627739 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e23413-951d-4689-9f74-c76d0b815b46 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:58:47.690715 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7744206a-c3e9-45f0-8fba-fae0d3e6fc90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:58:47.692891 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7744206a-c3e9-45f0-8fba-fae0d3e6fc90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:47.693176 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7744206a-c3e9-45f0-8fba-fae0d3e6fc90 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:47.693446 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7744206a-c3e9-45f0-8fba-fae0d3e6fc90 admin admin] PATH_INFO: `/v3/users/8d0af2b5f354439d92bfa317a0baefd5` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:47.694619 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7744206a-c3e9-45f0-8fba-fae0d3e6fc90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d0af2b5f354439d92bfa317a0baefd5)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:58:47.696650 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7744206a-c3e9-45f0-8fba-fae0d3e6fc90 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:58:47.719340 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:58:47.749459 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-7744206a-c3e9-45f0-8fba-fae0d3e6fc90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d0af2b5f354439d92bfa317a0baefd5'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:47.757383 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-7744206a-c3e9-45f0-8fba-fae0d3e6fc90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d0af2b5f354439d92bfa317a0baefd5'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:47.758140 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-7744206a-c3e9-45f0-8fba-fae0d3e6fc90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d0af2b5f354439d92bfa317a0baefd5'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:47.758745 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-7744206a-c3e9-45f0-8fba-fae0d3e6fc90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d0af2b5f354439d92bfa317a0baefd5'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:47.765704 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-7744206a-c3e9-45f0-8fba-fae0d3e6fc90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d0af2b5f354439d92bfa317a0baefd5'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:47.783310 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1328/2667] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:58:47 2022] DELETE /identity/v3/users/8d0af2b5f354439d92bfa317a0baefd5 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:58:47.956916 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977ffe7a-c5f7-48d3-b2d2-93b117297ad1 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:58:47.958478 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-977ffe7a-c5f7-48d3-b2d2-93b117297ad1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:58:47.980001 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977ffe7a-c5f7-48d3-b2d2-93b117297ad1 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:58:47.984273 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:58:48.047043 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29aad20-2141-468a-b717-035b8bcbb8ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:58:48.047708 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29aad20-2141-468a-b717-035b8bcbb8ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:48.047989 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29aad20-2141-468a-b717-035b8bcbb8ad admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:48.048265 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29aad20-2141-468a-b717-035b8bcbb8ad admin admin] PATH_INFO: `/v3/projects/204ecef1ba96491cbfb3e8176e6c22ec` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:48.049461 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29aad20-2141-468a-b717-035b8bcbb8ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=204ecef1ba96491cbfb3e8176e6c22ec)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:58:48.052599 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29aad20-2141-468a-b717-035b8bcbb8ad admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:58:48.116136 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-f29aad20-2141-468a-b717-035b8bcbb8ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '204ecef1ba96491cbfb3e8176e6c22ec', 'request_id': 'req-f29aad20-2141-468a-b717-035b8bcbb8ad'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:48.122592 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-f29aad20-2141-468a-b717-035b8bcbb8ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '204ecef1ba96491cbfb3e8176e6c22ec', 'request_id': 'req-f29aad20-2141-468a-b717-035b8bcbb8ad'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:48.128140 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-f29aad20-2141-468a-b717-035b8bcbb8ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '204ecef1ba96491cbfb3e8176e6c22ec', 'request_id': 'req-f29aad20-2141-468a-b717-035b8bcbb8ad'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:48.140764 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-f29aad20-2141-468a-b717-035b8bcbb8ad admin admin] The token cache is being invalidate because project 204ecef1ba96491cbfb3e8176e6c22ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:58:48.141052 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-f29aad20-2141-468a-b717-035b8bcbb8ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:48.141671 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-f29aad20-2141-468a-b717-035b8bcbb8ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:48.143176 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1340/2668] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:58:47 2022] DELETE /identity/v3/projects/204ecef1ba96491cbfb3e8176e6c22ec => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:58:48.148391 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc616913-277e-4f82-bb4c-785e6835cf35 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:48.148733 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc616913-277e-4f82-bb4c-785e6835cf35 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:48.149011 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc616913-277e-4f82-bb4c-785e6835cf35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:48.168127 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-fc616913-277e-4f82-bb4c-785e6835cf35 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:58:48.169293 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-fc616913-277e-4f82-bb4c-785e6835cf35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:58:48.228315 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1329/2669] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:58:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:58:48.236418 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff9cd7-7ad0-47af-bfc9-0e12dd3907d4 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:48.236702 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff9cd7-7ad0-47af-bfc9-0e12dd3907d4 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:48.236973 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff9cd7-7ad0-47af-bfc9-0e12dd3907d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:48.257364 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-b1ff9cd7-7ad0-47af-bfc9-0e12dd3907d4 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:58:48.258570 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-b1ff9cd7-7ad0-47af-bfc9-0e12dd3907d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:58:48.266759 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1341/2670] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:58:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:58:48.270849 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc616913-277e-4f82-bb4c-785e6835cf35 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:58:48.280565 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc616913-277e-4f82-bb4c-785e6835cf35 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:58:48.285443 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f314b6dd-4bcf-42d0-ae65-8e442958a6aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:58:48.286192 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f314b6dd-4bcf-42d0-ae65-8e442958a6aa admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:48.286471 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f314b6dd-4bcf-42d0-ae65-8e442958a6aa admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:48.286736 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f314b6dd-4bcf-42d0-ae65-8e442958a6aa admin admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:48.287380 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f314b6dd-4bcf-42d0-ae65-8e442958a6aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:58:48.287713 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f314b6dd-4bcf-42d0-ae65-8e442958a6aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:58:48.289320 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f314b6dd-4bcf-42d0-ae65-8e442958a6aa admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:58:48.301877 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1330/2671] 10.222.0.15 () {58 vars in 1153 bytes} [Sun Sep 11 10:58:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:58:48.306402 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ff9cd7-7ad0-47af-bfc9-0e12dd3907d4 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:58:48.315711 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ff9cd7-7ad0-47af-bfc9-0e12dd3907d4 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:58:48.320929 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2fa835-ab9c-4faa-8d6e-746db786178a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:58:48.321526 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2fa835-ab9c-4faa-8d6e-746db786178a admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:48.321807 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2fa835-ab9c-4faa-8d6e-746db786178a admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:48.322080 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2fa835-ab9c-4faa-8d6e-746db786178a admin admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:48.322705 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2fa835-ab9c-4faa-8d6e-746db786178a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:58:48.324679 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2fa835-ab9c-4faa-8d6e-746db786178a admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:58:48.363814 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1342/2672] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:58:48 2022] POST /identity/v3/projects => generated 401 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:58:48.366366 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f314b6dd-4bcf-42d0-ae65-8e442958a6aa admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:58:48.376356 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f314b6dd-4bcf-42d0-ae65-8e442958a6aa admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:58:48.432118 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1bef55-6683-4db7-aed0-5f90e0bb8072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:58:48.432720 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1bef55-6683-4db7-aed0-5f90e0bb8072 admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:48.433003 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1bef55-6683-4db7-aed0-5f90e0bb8072 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:48.433276 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1bef55-6683-4db7-aed0-5f90e0bb8072 admin admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:48.433901 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1bef55-6683-4db7-aed0-5f90e0bb8072 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:58:48.435849 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1bef55-6683-4db7-aed0-5f90e0bb8072 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:58:48.457117 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1331/2673] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:58:48 2022] POST /identity/v3/users => generated 301 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:58:48.461192 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2fa835-ab9c-4faa-8d6e-746db786178a admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:58:48.471222 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2fa835-ab9c-4faa-8d6e-746db786178a admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:58:48.476374 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd118750-3b13-4f9f-a525-9cbe750c5142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:58:48.476967 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd118750-3b13-4f9f-a525-9cbe750c5142 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:48.477256 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd118750-3b13-4f9f-a525-9cbe750c5142 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:48.477517 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd118750-3b13-4f9f-a525-9cbe750c5142 admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:48.478149 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd118750-3b13-4f9f-a525-9cbe750c5142 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:58:48.479841 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd118750-3b13-4f9f-a525-9cbe750c5142 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:58:48.490298 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1343/2674] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:58:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:58:48.494118 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1bef55-6683-4db7-aed0-5f90e0bb8072 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:58:48.503897 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1bef55-6683-4db7-aed0-5f90e0bb8072 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:58:48.508747 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5bf7e8-af4f-435e-8cd7-61f3e8694cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:58:48.509327 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5bf7e8-af4f-435e-8cd7-61f3e8694cf7 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:48.509602 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5bf7e8-af4f-435e-8cd7-61f3e8694cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:48.509870 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5bf7e8-af4f-435e-8cd7-61f3e8694cf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:48.510491 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5bf7e8-af4f-435e-8cd7-61f3e8694cf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:58:48.512230 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5bf7e8-af4f-435e-8cd7-61f3e8694cf7 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:58:48.523026 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1332/2675] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:58:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:58:48.527185 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd118750-3b13-4f9f-a525-9cbe750c5142 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:58:48.537906 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd118750-3b13-4f9f-a525-9cbe750c5142 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:58:48.542782 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0765c35d-880f-4bc7-aacf-8a058dc6640a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:58:48.543447 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0765c35d-880f-4bc7-aacf-8a058dc6640a admin admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:48.543754 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0765c35d-880f-4bc7-aacf-8a058dc6640a admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:48.544029 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0765c35d-880f-4bc7-aacf-8a058dc6640a admin admin] PATH_INFO: `/v3/projects/86333be5ce794b2d9471f13398718ab9/users/5573d70cef3a4b6db97f07c304e5d524/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:48.559216 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0765c35d-880f-4bc7-aacf-8a058dc6640a admin admin] RBAC: Authorizing `identity:create_grant(project_id=86333be5ce794b2d9471f13398718ab9, user_id=5573d70cef3a4b6db97f07c304e5d524, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:58:48.561901 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0765c35d-880f-4bc7-aacf-8a058dc6640a admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:58:48.574749 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1344/2676] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:58:48 2022] PUT /identity/v3/projects/86333be5ce794b2d9471f13398718ab9/users/5573d70cef3a4b6db97f07c304e5d524/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:58:48.585313 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d99bd2-a9c7-4543-9892-785868bb7601 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:48.585606 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d99bd2-a9c7-4543-9892-785868bb7601 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:48.585880 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d99bd2-a9c7-4543-9892-785868bb7601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:48.593834 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:58:48.617083 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-78d99bd2-a9c7-4543-9892-785868bb7601 None None] MFA Rules not processed for user `5573d70cef3a4b6db97f07c304e5d524`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:58:48.618336 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-78d99bd2-a9c7-4543-9892-785868bb7601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:58:48.681401 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1333/2677] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:58:48 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:58:48.685945 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0765c35d-880f-4bc7-aacf-8a058dc6640a admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:58:48.712441 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:58:48.749313 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0765c35d-880f-4bc7-aacf-8a058dc6640a admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:58:48.753959 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07884063-a1be-402f-87a9-9e7ae06c4516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:58:48.754558 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07884063-a1be-402f-87a9-9e7ae06c4516 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:48.754840 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07884063-a1be-402f-87a9-9e7ae06c4516 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:48.755113 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07884063-a1be-402f-87a9-9e7ae06c4516 admin admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:48.755772 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07884063-a1be-402f-87a9-9e7ae06c4516 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:58:48.757673 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07884063-a1be-402f-87a9-9e7ae06c4516 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:58:48.784359 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1345/2678] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:58:48 2022] POST /identity/v3/projects => generated 403 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:58:48.788999 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d99bd2-a9c7-4543-9892-785868bb7601 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:58:48.799376 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d99bd2-a9c7-4543-9892-785868bb7601 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:58:48.855922 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a133e4b8-483b-4221-83b4-0707ac5569a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:58:48.856526 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a133e4b8-483b-4221-83b4-0707ac5569a2 admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:48.856804 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a133e4b8-483b-4221-83b4-0707ac5569a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:48.857077 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a133e4b8-483b-4221-83b4-0707ac5569a2 admin admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:48.857698 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a133e4b8-483b-4221-83b4-0707ac5569a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:58:48.859635 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a133e4b8-483b-4221-83b4-0707ac5569a2 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:58:48.880492 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1334/2679] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:58:48 2022] POST /identity/v3/users => generated 308 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:58:48.884539 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07884063-a1be-402f-87a9-9e7ae06c4516 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:58:48.895689 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07884063-a1be-402f-87a9-9e7ae06c4516 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:58:48.900555 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eb3371-e709-4d80-b8d9-fb2170dd2f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:58:48.901154 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb3371-e709-4d80-b8d9-fb2170dd2f93 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:48.901433 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb3371-e709-4d80-b8d9-fb2170dd2f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:48.901735 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb3371-e709-4d80-b8d9-fb2170dd2f93 admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:48.902371 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eb3371-e709-4d80-b8d9-fb2170dd2f93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:58:48.903992 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eb3371-e709-4d80-b8d9-fb2170dd2f93 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:58:48.915218 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1346/2680] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:58:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:58:48.919536 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a133e4b8-483b-4221-83b4-0707ac5569a2 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:58:48.929627 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a133e4b8-483b-4221-83b4-0707ac5569a2 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:58:48.934201 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd98a42c-e328-4ec1-94dd-5512694cf481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:58:48.934847 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd98a42c-e328-4ec1-94dd-5512694cf481 admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:48.935129 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd98a42c-e328-4ec1-94dd-5512694cf481 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:48.935404 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd98a42c-e328-4ec1-94dd-5512694cf481 admin admin] PATH_INFO: `/v3/projects/4acf76cd165942fd9dd6edc60ead8cfa/users/0ac2cc6353904b8281a2c3ac8e6d7b2c/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:48.951511 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd98a42c-e328-4ec1-94dd-5512694cf481 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4acf76cd165942fd9dd6edc60ead8cfa, user_id=0ac2cc6353904b8281a2c3ac8e6d7b2c, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:58:48.954209 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd98a42c-e328-4ec1-94dd-5512694cf481 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:58:48.967115 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1335/2681] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:58:48 2022] PUT /identity/v3/projects/4acf76cd165942fd9dd6edc60ead8cfa/users/0ac2cc6353904b8281a2c3ac8e6d7b2c/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:58:48.971111 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eb3371-e709-4d80-b8d9-fb2170dd2f93 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:58:48.981578 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eb3371-e709-4d80-b8d9-fb2170dd2f93 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:58:49.037819 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66ec74c-885f-41f9-80e0-0f982f0d3c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:58:49.038436 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ec74c-885f-41f9-80e0-0f982f0d3c4c admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:49.038717 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ec74c-885f-41f9-80e0-0f982f0d3c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:49.038992 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ec74c-885f-41f9-80e0-0f982f0d3c4c admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:49.039659 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66ec74c-885f-41f9-80e0-0f982f0d3c4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:58:49.041260 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66ec74c-885f-41f9-80e0-0f982f0d3c4c admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:58:49.051107 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1347/2682] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:58:48 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:58:49.055209 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd98a42c-e328-4ec1-94dd-5512694cf481 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:58:49.065493 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd98a42c-e328-4ec1-94dd-5512694cf481 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:58:49.070732 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61995e7-5566-4486-a22b-1590095fb33e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:58:49.071383 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61995e7-5566-4486-a22b-1590095fb33e admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:49.071705 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61995e7-5566-4486-a22b-1590095fb33e admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:49.071980 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61995e7-5566-4486-a22b-1590095fb33e admin admin] PATH_INFO: `/v3/projects/4acf76cd165942fd9dd6edc60ead8cfa/users/0ac2cc6353904b8281a2c3ac8e6d7b2c/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:49.073883 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61995e7-5566-4486-a22b-1590095fb33e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4acf76cd165942fd9dd6edc60ead8cfa, user_id=0ac2cc6353904b8281a2c3ac8e6d7b2c, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:58:49.076553 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61995e7-5566-4486-a22b-1590095fb33e admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:58:49.087531 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1336/2683] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:58:49 2022] PUT /identity/v3/projects/4acf76cd165942fd9dd6edc60ead8cfa/users/0ac2cc6353904b8281a2c3ac8e6d7b2c/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:58:49.099227 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b666db8-0d57-4480-95c4-c01f20a8e694 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:49.099547 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b666db8-0d57-4480-95c4-c01f20a8e694 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:49.099828 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b666db8-0d57-4480-95c4-c01f20a8e694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:49.127291 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-3b666db8-0d57-4480-95c4-c01f20a8e694 None None] MFA Rules not processed for user `0ac2cc6353904b8281a2c3ac8e6d7b2c`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:58:49.128537 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-3b666db8-0d57-4480-95c4-c01f20a8e694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:58:49.196628 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1348/2684] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:58:49 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:58:49.207744 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61995e7-5566-4486-a22b-1590095fb33e admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:58:49.209478 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-f61995e7-5566-4486-a22b-1590095fb33e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:58:49.261126 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:58:49.267532 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61995e7-5566-4486-a22b-1590095fb33e admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:58:49.272090 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e20a9a-bcae-484a-b556-90efa1652083 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:58:49.272725 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e20a9a-bcae-484a-b556-90efa1652083 service cinder] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:49.273000 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e20a9a-bcae-484a-b556-90efa1652083 service cinder] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:49.273266 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e20a9a-bcae-484a-b556-90efa1652083 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:49.282450 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e20a9a-bcae-484a-b556-90efa1652083 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:58:49.283900 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e20a9a-bcae-484a-b556-90efa1652083 service cinder] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:58:49.293515 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1337/2685] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:58:49 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:58:49.443792 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b666db8-0d57-4480-95c4-c01f20a8e694 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:58:49.464035 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:58:49.506553 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b666db8-0d57-4480-95c4-c01f20a8e694 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:58:49.511060 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a3e8ab-022b-4286-93cf-cbd0b3f15452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:58:49.511732 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a3e8ab-022b-4286-93cf-cbd0b3f15452 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:49.512013 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a3e8ab-022b-4286-93cf-cbd0b3f15452 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:49.512290 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a3e8ab-022b-4286-93cf-cbd0b3f15452 admin admin] PATH_INFO: `/v3/users/5573d70cef3a4b6db97f07c304e5d524` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:49.513394 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a3e8ab-022b-4286-93cf-cbd0b3f15452 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5573d70cef3a4b6db97f07c304e5d524)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:58:49.515317 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a3e8ab-022b-4286-93cf-cbd0b3f15452 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:58:49.560137 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-95a3e8ab-022b-4286-93cf-cbd0b3f15452 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5573d70cef3a4b6db97f07c304e5d524'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:49.568140 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-95a3e8ab-022b-4286-93cf-cbd0b3f15452 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5573d70cef3a4b6db97f07c304e5d524'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:49.568757 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-95a3e8ab-022b-4286-93cf-cbd0b3f15452 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5573d70cef3a4b6db97f07c304e5d524'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:49.569383 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-95a3e8ab-022b-4286-93cf-cbd0b3f15452 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5573d70cef3a4b6db97f07c304e5d524'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:49.581118 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-95a3e8ab-022b-4286-93cf-cbd0b3f15452 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5573d70cef3a4b6db97f07c304e5d524'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:49.593134 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1349/2686] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:58:49 2022] DELETE /identity/v3/users/5573d70cef3a4b6db97f07c304e5d524 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:58:49.597831 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe5491-9a43-4e34-ad9e-f40630e2a145 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:49.598111 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe5491-9a43-4e34-ad9e-f40630e2a145 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:49.598379 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe5491-9a43-4e34-ad9e-f40630e2a145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:49.618188 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-d1fe5491-9a43-4e34-ad9e-f40630e2a145 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:58:49.619408 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-d1fe5491-9a43-4e34-ad9e-f40630e2a145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:58:49.679178 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1338/2687] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:58:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:58:49.689124 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a3e8ab-022b-4286-93cf-cbd0b3f15452 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:58:49.690550 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-95a3e8ab-022b-4286-93cf-cbd0b3f15452 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:58:49.750970 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a3e8ab-022b-4286-93cf-cbd0b3f15452 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:58:49.755760 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d15bf5-e729-4263-be8a-926317c0b128 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:58:49.756555 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d15bf5-e729-4263-be8a-926317c0b128 service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:49.756838 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d15bf5-e729-4263-be8a-926317c0b128 service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:49.757110 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d15bf5-e729-4263-be8a-926317c0b128 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:49.766206 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d15bf5-e729-4263-be8a-926317c0b128 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:58:49.767655 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d15bf5-e729-4263-be8a-926317c0b128 service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:58:49.777232 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1350/2688] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:58:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:58:50.048574 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fe5491-9a43-4e34-ad9e-f40630e2a145 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:58:50.050093 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-d1fe5491-9a43-4e34-ad9e-f40630e2a145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:58:50.063645 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fe5491-9a43-4e34-ad9e-f40630e2a145 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:58:50.068466 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee40f424-c400-477c-8eaf-5947dd991bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:58:50.069170 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee40f424-c400-477c-8eaf-5947dd991bbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:50.069445 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee40f424-c400-477c-8eaf-5947dd991bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:50.069712 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee40f424-c400-477c-8eaf-5947dd991bbc admin admin] PATH_INFO: `/v3/projects/86333be5ce794b2d9471f13398718ab9` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:50.070826 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee40f424-c400-477c-8eaf-5947dd991bbc admin admin] RBAC: Authorizing `identity:delete_project(project_id=86333be5ce794b2d9471f13398718ab9)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:58:50.073164 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee40f424-c400-477c-8eaf-5947dd991bbc admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:58:50.123016 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-ee40f424-c400-477c-8eaf-5947dd991bbc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86333be5ce794b2d9471f13398718ab9', 'request_id': 'req-ee40f424-c400-477c-8eaf-5947dd991bbc'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:50.130035 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-ee40f424-c400-477c-8eaf-5947dd991bbc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86333be5ce794b2d9471f13398718ab9', 'request_id': 'req-ee40f424-c400-477c-8eaf-5947dd991bbc'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:50.137613 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-ee40f424-c400-477c-8eaf-5947dd991bbc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86333be5ce794b2d9471f13398718ab9', 'request_id': 'req-ee40f424-c400-477c-8eaf-5947dd991bbc'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:50.148919 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-ee40f424-c400-477c-8eaf-5947dd991bbc admin admin] The token cache is being invalidate because project 86333be5ce794b2d9471f13398718ab9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90653) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:58:50.149206 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-ee40f424-c400-477c-8eaf-5947dd991bbc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:50.149918 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-ee40f424-c400-477c-8eaf-5947dd991bbc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:50.151370 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1339/2689] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:58:50 2022] DELETE /identity/v3/projects/86333be5ce794b2d9471f13398718ab9 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:58:50.155475 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d15bf5-e729-4263-be8a-926317c0b128 service neutron] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:58:50.165979 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d15bf5-e729-4263-be8a-926317c0b128 service neutron] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:58:50.208486 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:58:50.227563 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4266d3-ea65-4ee0-bb6e-49b19db8238b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:58:50.228203 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4266d3-ea65-4ee0-bb6e-49b19db8238b admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:50.228481 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4266d3-ea65-4ee0-bb6e-49b19db8238b admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:50.228752 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4266d3-ea65-4ee0-bb6e-49b19db8238b admin admin] PATH_INFO: `/v3/users/0ac2cc6353904b8281a2c3ac8e6d7b2c` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:50.229854 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4266d3-ea65-4ee0-bb6e-49b19db8238b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ac2cc6353904b8281a2c3ac8e6d7b2c)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:58:50.231812 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4266d3-ea65-4ee0-bb6e-49b19db8238b admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:58:50.277467 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-0a4266d3-ea65-4ee0-bb6e-49b19db8238b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ac2cc6353904b8281a2c3ac8e6d7b2c'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:50.285191 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-0a4266d3-ea65-4ee0-bb6e-49b19db8238b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ac2cc6353904b8281a2c3ac8e6d7b2c'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:50.286539 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-0a4266d3-ea65-4ee0-bb6e-49b19db8238b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ac2cc6353904b8281a2c3ac8e6d7b2c'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:50.287148 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-0a4266d3-ea65-4ee0-bb6e-49b19db8238b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ac2cc6353904b8281a2c3ac8e6d7b2c'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:50.299300 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-0a4266d3-ea65-4ee0-bb6e-49b19db8238b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ac2cc6353904b8281a2c3ac8e6d7b2c'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:50.311417 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1351/2690] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:58:50 2022] DELETE /identity/v3/users/0ac2cc6353904b8281a2c3ac8e6d7b2c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:58:50.578722 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee40f424-c400-477c-8eaf-5947dd991bbc admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:58:50.580203 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-ee40f424-c400-477c-8eaf-5947dd991bbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:58:50.593877 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee40f424-c400-477c-8eaf-5947dd991bbc admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:58:50.651374 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7f78ac-e7dc-4ef3-8427-2550cf5b6194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:58:50.652049 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7f78ac-e7dc-4ef3-8427-2550cf5b6194 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:50.652304 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7f78ac-e7dc-4ef3-8427-2550cf5b6194 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:50.652576 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7f78ac-e7dc-4ef3-8427-2550cf5b6194 admin admin] PATH_INFO: `/v3/projects/4acf76cd165942fd9dd6edc60ead8cfa` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:50.653679 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7f78ac-e7dc-4ef3-8427-2550cf5b6194 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4acf76cd165942fd9dd6edc60ead8cfa)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:58:50.655615 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7f78ac-e7dc-4ef3-8427-2550cf5b6194 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:58:50.671137 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:58:50.703652 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-0f7f78ac-e7dc-4ef3-8427-2550cf5b6194 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4acf76cd165942fd9dd6edc60ead8cfa', 'request_id': 'req-0f7f78ac-e7dc-4ef3-8427-2550cf5b6194'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:50.710520 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-0f7f78ac-e7dc-4ef3-8427-2550cf5b6194 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4acf76cd165942fd9dd6edc60ead8cfa', 'request_id': 'req-0f7f78ac-e7dc-4ef3-8427-2550cf5b6194'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:50.718719 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-0f7f78ac-e7dc-4ef3-8427-2550cf5b6194 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4acf76cd165942fd9dd6edc60ead8cfa', 'request_id': 'req-0f7f78ac-e7dc-4ef3-8427-2550cf5b6194'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:50.729394 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-0f7f78ac-e7dc-4ef3-8427-2550cf5b6194 admin admin] The token cache is being invalidate because project 4acf76cd165942fd9dd6edc60ead8cfa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90653) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:58:50.729680 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-0f7f78ac-e7dc-4ef3-8427-2550cf5b6194 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:50.730309 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-0f7f78ac-e7dc-4ef3-8427-2550cf5b6194 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:50.731809 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1340/2691] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:58:50 2022] DELETE /identity/v3/projects/4acf76cd165942fd9dd6edc60ead8cfa => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:58:50.736893 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedc35f5-cbdb-466a-a593-deafa39c39af None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:50.737180 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedc35f5-cbdb-466a-a593-deafa39c39af None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:50.737456 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedc35f5-cbdb-466a-a593-deafa39c39af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:50.757088 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-dedc35f5-cbdb-466a-a593-deafa39c39af None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:58:50.758281 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-dedc35f5-cbdb-466a-a593-deafa39c39af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:58:50.814727 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1352/2692] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:58:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:58:50.822982 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f6e0b-fb65-420c-aad5-bf65bcb6417f None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:50.823205 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f6e0b-fb65-420c-aad5-bf65bcb6417f None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:50.823518 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f6e0b-fb65-420c-aad5-bf65bcb6417f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:50.844314 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-7c7f6e0b-fb65-420c-aad5-bf65bcb6417f None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:58:50.845531 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-7c7f6e0b-fb65-420c-aad5-bf65bcb6417f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:58:50.854113 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1341/2693] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:58:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:58:50.857577 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedc35f5-cbdb-466a-a593-deafa39c39af None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:58:50.867529 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedc35f5-cbdb-466a-a593-deafa39c39af None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:58:50.872039 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5622c1-2bb8-4946-8dc2-67d29b620222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:58:50.872633 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5622c1-2bb8-4946-8dc2-67d29b620222 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:50.872913 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5622c1-2bb8-4946-8dc2-67d29b620222 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:50.873191 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5622c1-2bb8-4946-8dc2-67d29b620222 admin admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:50.873831 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5622c1-2bb8-4946-8dc2-67d29b620222 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:58:50.874130 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5622c1-2bb8-4946-8dc2-67d29b620222 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:58:50.875758 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5622c1-2bb8-4946-8dc2-67d29b620222 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:58:50.887597 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1353/2694] 10.222.0.15 () {58 vars in 1153 bytes} [Sun Sep 11 10:58:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:58:50.891981 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7f6e0b-fb65-420c-aad5-bf65bcb6417f None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:58:50.902038 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7f6e0b-fb65-420c-aad5-bf65bcb6417f None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:58:50.906751 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b111c15d-e5bf-4f02-80c5-912ac91f03f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:58:50.907345 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b111c15d-e5bf-4f02-80c5-912ac91f03f1 admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:50.907659 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b111c15d-e5bf-4f02-80c5-912ac91f03f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:50.907936 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b111c15d-e5bf-4f02-80c5-912ac91f03f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:50.908570 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b111c15d-e5bf-4f02-80c5-912ac91f03f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:58:50.910685 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b111c15d-e5bf-4f02-80c5-912ac91f03f1 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:58:50.938496 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1342/2695] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:58:50 2022] POST /identity/v3/projects => generated 401 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:58:50.943386 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5622c1-2bb8-4946-8dc2-67d29b620222 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:58:50.953514 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5622c1-2bb8-4946-8dc2-67d29b620222 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:58:50.996528 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:58:51.010570 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0532cf5a-6fcb-44d0-bb3d-a762a256d565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:58:51.011186 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0532cf5a-6fcb-44d0-bb3d-a762a256d565 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:51.011553 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0532cf5a-6fcb-44d0-bb3d-a762a256d565 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:51.011862 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0532cf5a-6fcb-44d0-bb3d-a762a256d565 admin admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:51.012499 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0532cf5a-6fcb-44d0-bb3d-a762a256d565 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:58:51.014476 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0532cf5a-6fcb-44d0-bb3d-a762a256d565 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:58:51.034543 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1354/2696] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:58:50 2022] POST /identity/v3/users => generated 301 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 10:58:51.038834 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b111c15d-e5bf-4f02-80c5-912ac91f03f1 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:58:51.049151 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b111c15d-e5bf-4f02-80c5-912ac91f03f1 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:58:51.054365 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4a7f2b-ba90-44cb-8c29-0f8d845ef7d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:58:51.054955 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a7f2b-ba90-44cb-8c29-0f8d845ef7d2 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:51.055237 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a7f2b-ba90-44cb-8c29-0f8d845ef7d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:51.055543 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a7f2b-ba90-44cb-8c29-0f8d845ef7d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:51.056178 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4a7f2b-ba90-44cb-8c29-0f8d845ef7d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:58:51.057768 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4a7f2b-ba90-44cb-8c29-0f8d845ef7d2 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:58:51.068106 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1343/2697] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:58:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:58:51.072008 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0532cf5a-6fcb-44d0-bb3d-a762a256d565 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:58:51.082327 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0532cf5a-6fcb-44d0-bb3d-a762a256d565 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:58:51.086946 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3eda84-12db-49cd-9514-599d5fb9ffdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:58:51.087579 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3eda84-12db-49cd-9514-599d5fb9ffdf admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:51.087864 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3eda84-12db-49cd-9514-599d5fb9ffdf admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:51.088139 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3eda84-12db-49cd-9514-599d5fb9ffdf admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:51.088783 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3eda84-12db-49cd-9514-599d5fb9ffdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:58:51.090375 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3eda84-12db-49cd-9514-599d5fb9ffdf admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:58:51.100844 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1355/2698] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:58:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 10:58:51.104887 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4a7f2b-ba90-44cb-8c29-0f8d845ef7d2 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:58:51.115800 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4a7f2b-ba90-44cb-8c29-0f8d845ef7d2 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:58:51.120376 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02400379-5e04-450d-bc4b-d52b180f2f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:58:51.121019 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02400379-5e04-450d-bc4b-d52b180f2f12 admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:51.121425 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02400379-5e04-450d-bc4b-d52b180f2f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:51.121545 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02400379-5e04-450d-bc4b-d52b180f2f12 admin admin] PATH_INFO: `/v3/projects/92acb89fdf5041618baee280b3b6418c/users/c9632b6ae19b4976b0856dd38fcdf1ad/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:51.136771 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02400379-5e04-450d-bc4b-d52b180f2f12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92acb89fdf5041618baee280b3b6418c, user_id=c9632b6ae19b4976b0856dd38fcdf1ad, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:58:51.139383 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02400379-5e04-450d-bc4b-d52b180f2f12 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:58:51.152267 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1344/2699] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:58:51 2022] PUT /identity/v3/projects/92acb89fdf5041618baee280b3b6418c/users/c9632b6ae19b4976b0856dd38fcdf1ad/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:58:51.162834 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d73b449-2287-4e11-adda-242785e53c65 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:51.163316 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d73b449-2287-4e11-adda-242785e53c65 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:51.163460 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d73b449-2287-4e11-adda-242785e53c65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:51.190769 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-4d73b449-2287-4e11-adda-242785e53c65 None None] MFA Rules not processed for user `c9632b6ae19b4976b0856dd38fcdf1ad`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:58:51.192745 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-4d73b449-2287-4e11-adda-242785e53c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:58:51.255067 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1356/2700] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:58:51 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:58:51.259504 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02400379-5e04-450d-bc4b-d52b180f2f12 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:58:51.320427 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02400379-5e04-450d-bc4b-d52b180f2f12 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:58:51.325050 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4d8fc6-4180-4aff-8527-12319521147b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:58:51.325635 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d8fc6-4180-4aff-8527-12319521147b admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:51.325915 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d8fc6-4180-4aff-8527-12319521147b admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:51.326188 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d8fc6-4180-4aff-8527-12319521147b admin admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:51.326827 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4d8fc6-4180-4aff-8527-12319521147b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:58:51.328774 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4d8fc6-4180-4aff-8527-12319521147b admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:58:51.355157 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1345/2701] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:58:51 2022] POST /identity/v3/projects => generated 399 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:58:51.359171 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d73b449-2287-4e11-adda-242785e53c65 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:58:51.370741 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d73b449-2287-4e11-adda-242785e53c65 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:58:51.426261 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed57b908-b7d0-45f0-b6d2-ce21080d2084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:58:51.427010 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed57b908-b7d0-45f0-b6d2-ce21080d2084 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:51.427291 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed57b908-b7d0-45f0-b6d2-ce21080d2084 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:51.427670 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed57b908-b7d0-45f0-b6d2-ce21080d2084 admin admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:51.428289 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed57b908-b7d0-45f0-b6d2-ce21080d2084 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:58:51.430202 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed57b908-b7d0-45f0-b6d2-ce21080d2084 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:58:51.450091 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1357/2702] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:58:51 2022] POST /identity/v3/users => generated 306 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:58:51.454127 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4d8fc6-4180-4aff-8527-12319521147b admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:58:51.464528 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4d8fc6-4180-4aff-8527-12319521147b admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:58:51.469108 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753cc736-fefe-4a2e-8921-bbc31cb887f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:58:51.469721 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753cc736-fefe-4a2e-8921-bbc31cb887f2 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:51.469996 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753cc736-fefe-4a2e-8921-bbc31cb887f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:51.470290 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753cc736-fefe-4a2e-8921-bbc31cb887f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:51.470914 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753cc736-fefe-4a2e-8921-bbc31cb887f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:58:51.473081 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753cc736-fefe-4a2e-8921-bbc31cb887f2 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:58:51.483255 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1346/2703] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:58:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 10:58:51.487557 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed57b908-b7d0-45f0-b6d2-ce21080d2084 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:58:51.498000 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed57b908-b7d0-45f0-b6d2-ce21080d2084 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:58:51.502861 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92230c22-07d0-4354-9390-650010f09b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:58:51.503606 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92230c22-07d0-4354-9390-650010f09b75 admin admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:51.503928 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92230c22-07d0-4354-9390-650010f09b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:51.504367 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92230c22-07d0-4354-9390-650010f09b75 admin admin] PATH_INFO: `/v3/projects/72a8d0769fd74c20bff72d5a890ff28e/users/db7fe773ea3043ff9e6f7af325a4c581/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:51.518759 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92230c22-07d0-4354-9390-650010f09b75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72a8d0769fd74c20bff72d5a890ff28e, user_id=db7fe773ea3043ff9e6f7af325a4c581, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:58:51.521741 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92230c22-07d0-4354-9390-650010f09b75 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:58:51.534484 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1358/2704] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:58:51 2022] PUT /identity/v3/projects/72a8d0769fd74c20bff72d5a890ff28e/users/db7fe773ea3043ff9e6f7af325a4c581/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:58:51.539114 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753cc736-fefe-4a2e-8921-bbc31cb887f2 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:58:51.549123 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753cc736-fefe-4a2e-8921-bbc31cb887f2 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:58:51.558817 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:58:51.607549 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d1120b-9f07-4fed-8bf3-8e56d05deaba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:58:51.608080 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d1120b-9f07-4fed-8bf3-8e56d05deaba admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:51.608363 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d1120b-9f07-4fed-8bf3-8e56d05deaba admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:51.608636 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d1120b-9f07-4fed-8bf3-8e56d05deaba admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:51.609264 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d1120b-9f07-4fed-8bf3-8e56d05deaba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:58:51.611108 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d1120b-9f07-4fed-8bf3-8e56d05deaba admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:58:51.621755 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1347/2705] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:58:51 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:58:51.625766 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92230c22-07d0-4354-9390-650010f09b75 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:58:51.636260 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92230c22-07d0-4354-9390-650010f09b75 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:58:51.641022 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7af1f7-2a30-4bd2-a794-d8df3691a206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:58:51.641672 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7af1f7-2a30-4bd2-a794-d8df3691a206 admin admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:51.641951 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7af1f7-2a30-4bd2-a794-d8df3691a206 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:51.642227 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7af1f7-2a30-4bd2-a794-d8df3691a206 admin admin] PATH_INFO: `/v3/projects/72a8d0769fd74c20bff72d5a890ff28e/users/db7fe773ea3043ff9e6f7af325a4c581/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:51.644158 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7af1f7-2a30-4bd2-a794-d8df3691a206 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72a8d0769fd74c20bff72d5a890ff28e, user_id=db7fe773ea3043ff9e6f7af325a4c581, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:58:51.647024 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7af1f7-2a30-4bd2-a794-d8df3691a206 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:58:51.661151 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1359/2706] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:58:51 2022] PUT /identity/v3/projects/72a8d0769fd74c20bff72d5a890ff28e/users/db7fe773ea3043ff9e6f7af325a4c581/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:58:51.672275 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705e0371-c6b8-4a56-bd63-f9aa2d34e6f6 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:51.672784 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705e0371-c6b8-4a56-bd63-f9aa2d34e6f6 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:51.672897 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705e0371-c6b8-4a56-bd63-f9aa2d34e6f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:51.700587 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-705e0371-c6b8-4a56-bd63-f9aa2d34e6f6 None None] MFA Rules not processed for user `db7fe773ea3043ff9e6f7af325a4c581`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:58:51.701799 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-705e0371-c6b8-4a56-bd63-f9aa2d34e6f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:58:51.770960 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1348/2707] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:58:51 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:58:51.781807 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7af1f7-2a30-4bd2-a794-d8df3691a206 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:58:51.783621 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-5f7af1f7-2a30-4bd2-a794-d8df3691a206 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:58:51.795097 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:58:51.842035 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7af1f7-2a30-4bd2-a794-d8df3691a206 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:58:51.845671 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b74f51-816a-4926-ba0f-f7a017146911 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:58:51.846273 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b74f51-816a-4926-ba0f-f7a017146911 service cinder] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:51.846551 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b74f51-816a-4926-ba0f-f7a017146911 service cinder] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:51.846823 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b74f51-816a-4926-ba0f-f7a017146911 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:51.856348 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b74f51-816a-4926-ba0f-f7a017146911 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:58:51.857869 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b74f51-816a-4926-ba0f-f7a017146911 service cinder] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:58:51.867765 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1360/2708] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:58:51 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:58:51.944511 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705e0371-c6b8-4a56-bd63-f9aa2d34e6f6 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:58:51.954263 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705e0371-c6b8-4a56-bd63-f9aa2d34e6f6 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:58:51.958583 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22503db0-7ef4-4d55-af2e-93064e347b62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:58:51.959177 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22503db0-7ef4-4d55-af2e-93064e347b62 service cinder] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:51.959495 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22503db0-7ef4-4d55-af2e-93064e347b62 service cinder] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:51.959775 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22503db0-7ef4-4d55-af2e-93064e347b62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:52.016029 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22503db0-7ef4-4d55-af2e-93064e347b62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:58:52.017548 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22503db0-7ef4-4d55-af2e-93064e347b62 service cinder] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:58:52.027062 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1349/2709] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:58:51 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 10:58:56.074920 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b74f51-816a-4926-ba0f-f7a017146911 service cinder] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:58:56.137367 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b74f51-816a-4926-ba0f-f7a017146911 service cinder] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:58:56.142010 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a77b1f-96ff-4419-b245-13117841fba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:58:56.142639 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a77b1f-96ff-4419-b245-13117841fba7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:56.142930 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a77b1f-96ff-4419-b245-13117841fba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:56.143200 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a77b1f-96ff-4419-b245-13117841fba7 admin admin] PATH_INFO: `/v3/users/c9632b6ae19b4976b0856dd38fcdf1ad` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:56.144409 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a77b1f-96ff-4419-b245-13117841fba7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9632b6ae19b4976b0856dd38fcdf1ad)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:58:56.146569 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a77b1f-96ff-4419-b245-13117841fba7 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:58:56.191618 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-f0a77b1f-96ff-4419-b245-13117841fba7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9632b6ae19b4976b0856dd38fcdf1ad'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:56.198663 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-f0a77b1f-96ff-4419-b245-13117841fba7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9632b6ae19b4976b0856dd38fcdf1ad'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:56.199261 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-f0a77b1f-96ff-4419-b245-13117841fba7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9632b6ae19b4976b0856dd38fcdf1ad'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:56.199898 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-f0a77b1f-96ff-4419-b245-13117841fba7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9632b6ae19b4976b0856dd38fcdf1ad'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:56.213332 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-f0a77b1f-96ff-4419-b245-13117841fba7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9632b6ae19b4976b0856dd38fcdf1ad'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:56.225910 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1361/2710] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:58:56 2022] DELETE /identity/v3/users/c9632b6ae19b4976b0856dd38fcdf1ad => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:58:56.230426 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fea4db-6b6e-4c4b-b0c3-15c40f7fc3d9 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:56.230920 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fea4db-6b6e-4c4b-b0c3-15c40f7fc3d9 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:56.231072 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fea4db-6b6e-4c4b-b0c3-15c40f7fc3d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:56.249754 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-62fea4db-6b6e-4c4b-b0c3-15c40f7fc3d9 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:58:56.250929 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-62fea4db-6b6e-4c4b-b0c3-15c40f7fc3d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:58:56.279717 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:58:56.311326 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1350/2711] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:58:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:58:56.321387 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a77b1f-96ff-4419-b245-13117841fba7 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:58:56.322895 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-f0a77b1f-96ff-4419-b245-13117841fba7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:58:56.365074 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:58:56.391042 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a77b1f-96ff-4419-b245-13117841fba7 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:58:56.394736 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f6394f-663e-4196-a25c-34bee6dd8649 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:58:56.396069 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f6394f-663e-4196-a25c-34bee6dd8649 service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:56.396478 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f6394f-663e-4196-a25c-34bee6dd8649 service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:56.396756 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f6394f-663e-4196-a25c-34bee6dd8649 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:56.406167 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f6394f-663e-4196-a25c-34bee6dd8649 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:58:56.407688 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f6394f-663e-4196-a25c-34bee6dd8649 service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:58:56.419121 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1362/2712] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:58:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:58:56.689359 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fea4db-6b6e-4c4b-b0c3-15c40f7fc3d9 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:58:56.690858 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-62fea4db-6b6e-4c4b-b0c3-15c40f7fc3d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:58:56.705327 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fea4db-6b6e-4c4b-b0c3-15c40f7fc3d9 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:58:56.711224 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b2f631-529b-47e2-a7a5-95bc77f9c67f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:58:56.712135 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b2f631-529b-47e2-a7a5-95bc77f9c67f admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:56.712410 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b2f631-529b-47e2-a7a5-95bc77f9c67f admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:56.712679 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b2f631-529b-47e2-a7a5-95bc77f9c67f admin admin] PATH_INFO: `/v3/projects/92acb89fdf5041618baee280b3b6418c` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:56.713797 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b2f631-529b-47e2-a7a5-95bc77f9c67f admin admin] RBAC: Authorizing `identity:delete_project(project_id=92acb89fdf5041618baee280b3b6418c)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:58:56.715719 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b2f631-529b-47e2-a7a5-95bc77f9c67f admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:58:56.761772 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-68b2f631-529b-47e2-a7a5-95bc77f9c67f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92acb89fdf5041618baee280b3b6418c', 'request_id': 'req-68b2f631-529b-47e2-a7a5-95bc77f9c67f'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:56.767799 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-68b2f631-529b-47e2-a7a5-95bc77f9c67f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92acb89fdf5041618baee280b3b6418c', 'request_id': 'req-68b2f631-529b-47e2-a7a5-95bc77f9c67f'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:56.776538 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-68b2f631-529b-47e2-a7a5-95bc77f9c67f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92acb89fdf5041618baee280b3b6418c', 'request_id': 'req-68b2f631-529b-47e2-a7a5-95bc77f9c67f'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:56.787234 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-68b2f631-529b-47e2-a7a5-95bc77f9c67f admin admin] The token cache is being invalidate because project 92acb89fdf5041618baee280b3b6418c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90653) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:58:56.787567 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-68b2f631-529b-47e2-a7a5-95bc77f9c67f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:56.788196 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-68b2f631-529b-47e2-a7a5-95bc77f9c67f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:56.789654 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1351/2713] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:58:56 2022] DELETE /identity/v3/projects/92acb89fdf5041618baee280b3b6418c => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:58:56.793639 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f6394f-663e-4196-a25c-34bee6dd8649 service neutron] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:58:56.803270 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f6394f-663e-4196-a25c-34bee6dd8649 service neutron] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:58:56.859459 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f4cf6d-9e38-499d-92fd-29fdd294a380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:58:56.860080 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f4cf6d-9e38-499d-92fd-29fdd294a380 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:56.860619 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f4cf6d-9e38-499d-92fd-29fdd294a380 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:56.860892 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f4cf6d-9e38-499d-92fd-29fdd294a380 admin admin] PATH_INFO: `/v3/users/db7fe773ea3043ff9e6f7af325a4c581` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:56.862006 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f4cf6d-9e38-499d-92fd-29fdd294a380 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db7fe773ea3043ff9e6f7af325a4c581)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:58:56.864159 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f4cf6d-9e38-499d-92fd-29fdd294a380 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:58:56.908288 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-94f4cf6d-9e38-499d-92fd-29fdd294a380 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db7fe773ea3043ff9e6f7af325a4c581'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:56.915584 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-94f4cf6d-9e38-499d-92fd-29fdd294a380 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db7fe773ea3043ff9e6f7af325a4c581'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:56.916208 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-94f4cf6d-9e38-499d-92fd-29fdd294a380 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db7fe773ea3043ff9e6f7af325a4c581'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:56.916795 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-94f4cf6d-9e38-499d-92fd-29fdd294a380 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db7fe773ea3043ff9e6f7af325a4c581'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:56.928972 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-94f4cf6d-9e38-499d-92fd-29fdd294a380 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db7fe773ea3043ff9e6f7af325a4c581'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:56.940647 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1363/2714] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:58:56 2022] DELETE /identity/v3/users/db7fe773ea3043ff9e6f7af325a4c581 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:58:57.207482 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b2f631-529b-47e2-a7a5-95bc77f9c67f admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:58:57.209094 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-68b2f631-529b-47e2-a7a5-95bc77f9c67f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:58:57.222253 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b2f631-529b-47e2-a7a5-95bc77f9c67f admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:58:57.278477 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d5aafa-d37e-41c1-a60b-f229f195411a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:58:57.279122 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d5aafa-d37e-41c1-a60b-f229f195411a admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:57.279382 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d5aafa-d37e-41c1-a60b-f229f195411a admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:57.279682 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d5aafa-d37e-41c1-a60b-f229f195411a admin admin] PATH_INFO: `/v3/projects/72a8d0769fd74c20bff72d5a890ff28e` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:57.280807 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d5aafa-d37e-41c1-a60b-f229f195411a admin admin] RBAC: Authorizing `identity:delete_project(project_id=72a8d0769fd74c20bff72d5a890ff28e)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:58:57.282708 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d5aafa-d37e-41c1-a60b-f229f195411a admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:58:57.330685 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-06d5aafa-d37e-41c1-a60b-f229f195411a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72a8d0769fd74c20bff72d5a890ff28e', 'request_id': 'req-06d5aafa-d37e-41c1-a60b-f229f195411a'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:57.336935 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-06d5aafa-d37e-41c1-a60b-f229f195411a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72a8d0769fd74c20bff72d5a890ff28e', 'request_id': 'req-06d5aafa-d37e-41c1-a60b-f229f195411a'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:57.340558 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:58:57.344111 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-06d5aafa-d37e-41c1-a60b-f229f195411a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72a8d0769fd74c20bff72d5a890ff28e', 'request_id': 'req-06d5aafa-d37e-41c1-a60b-f229f195411a'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:57.356610 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-06d5aafa-d37e-41c1-a60b-f229f195411a admin admin] The token cache is being invalidate because project 72a8d0769fd74c20bff72d5a890ff28e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90653) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:58:57.356894 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-06d5aafa-d37e-41c1-a60b-f229f195411a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:57.357519 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-06d5aafa-d37e-41c1-a60b-f229f195411a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:57.358972 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1352/2715] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:58:57 2022] DELETE /identity/v3/projects/72a8d0769fd74c20bff72d5a890ff28e => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:58:57.364748 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026646c-8613-4544-aa70-374b442893ed None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:57.365213 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026646c-8613-4544-aa70-374b442893ed None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:57.365469 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026646c-8613-4544-aa70-374b442893ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:57.384996 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-3026646c-8613-4544-aa70-374b442893ed None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:58:57.386181 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-3026646c-8613-4544-aa70-374b442893ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:58:57.431880 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:58:57.446628 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1364/2716] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:58:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:58:57.454246 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe27bce-b1cf-45ec-9b05-ce7f1bf834ac None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:57.454526 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe27bce-b1cf-45ec-9b05-ce7f1bf834ac None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:57.454795 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe27bce-b1cf-45ec-9b05-ce7f1bf834ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:57.476255 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-7fe27bce-b1cf-45ec-9b05-ce7f1bf834ac None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:58:57.477440 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-7fe27bce-b1cf-45ec-9b05-ce7f1bf834ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:58:57.485685 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1353/2717] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:58:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:58:57.489859 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026646c-8613-4544-aa70-374b442893ed None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:58:57.499483 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026646c-8613-4544-aa70-374b442893ed None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:58:57.503977 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a807dab-8cac-4df9-8b76-d95699c21767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:58:57.505124 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a807dab-8cac-4df9-8b76-d95699c21767 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:57.505387 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a807dab-8cac-4df9-8b76-d95699c21767 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:57.505727 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a807dab-8cac-4df9-8b76-d95699c21767 admin admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:57.506369 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a807dab-8cac-4df9-8b76-d95699c21767 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:58:57.506672 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a807dab-8cac-4df9-8b76-d95699c21767 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:58:57.508299 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a807dab-8cac-4df9-8b76-d95699c21767 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:58:57.519760 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1365/2718] 10.222.0.15 () {58 vars in 1153 bytes} [Sun Sep 11 10:58:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 10:58:57.524562 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe27bce-b1cf-45ec-9b05-ce7f1bf834ac None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:58:57.533761 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe27bce-b1cf-45ec-9b05-ce7f1bf834ac None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:58:57.538579 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3634256c-3cba-45ed-99f3-642e371a08df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:58:57.539166 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3634256c-3cba-45ed-99f3-642e371a08df admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:57.539486 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3634256c-3cba-45ed-99f3-642e371a08df admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:57.539763 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3634256c-3cba-45ed-99f3-642e371a08df admin admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:57.540382 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3634256c-3cba-45ed-99f3-642e371a08df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:58:57.542268 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3634256c-3cba-45ed-99f3-642e371a08df admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:58:57.569419 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1354/2719] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:58:57 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 10:58:57.573530 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a807dab-8cac-4df9-8b76-d95699c21767 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:58:57.583920 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a807dab-8cac-4df9-8b76-d95699c21767 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:58:57.637874 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83626d88-f35b-4fcc-be72-0d90b8bb945b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:58:57.638470 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83626d88-f35b-4fcc-be72-0d90b8bb945b admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:57.638745 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83626d88-f35b-4fcc-be72-0d90b8bb945b admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:57.639015 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83626d88-f35b-4fcc-be72-0d90b8bb945b admin admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:57.639655 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83626d88-f35b-4fcc-be72-0d90b8bb945b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:58:57.641557 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83626d88-f35b-4fcc-be72-0d90b8bb945b admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:58:57.663364 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1366/2720] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:58:57 2022] POST /identity/v3/users => generated 293 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:58:57.667582 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3634256c-3cba-45ed-99f3-642e371a08df admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:58:57.678255 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3634256c-3cba-45ed-99f3-642e371a08df admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:58:57.682800 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d45439-fa06-49dc-bb19-5ff57a3b08aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:58:57.683394 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d45439-fa06-49dc-bb19-5ff57a3b08aa admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:57.683838 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d45439-fa06-49dc-bb19-5ff57a3b08aa admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:57.684056 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d45439-fa06-49dc-bb19-5ff57a3b08aa admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:57.684700 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d45439-fa06-49dc-bb19-5ff57a3b08aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:58:57.686290 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d45439-fa06-49dc-bb19-5ff57a3b08aa admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:58:57.696816 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1355/2721] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:58:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:58:57.701475 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83626d88-f35b-4fcc-be72-0d90b8bb945b admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:58:57.712068 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83626d88-f35b-4fcc-be72-0d90b8bb945b admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:58:57.716757 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a63330d-6594-428a-ab06-bc783d631d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:58:57.717363 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a63330d-6594-428a-ab06-bc783d631d7f admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:57.717643 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a63330d-6594-428a-ab06-bc783d631d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:57.717912 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a63330d-6594-428a-ab06-bc783d631d7f admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:57.718549 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a63330d-6594-428a-ab06-bc783d631d7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:58:57.720187 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a63330d-6594-428a-ab06-bc783d631d7f admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:58:57.732278 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1367/2722] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:58:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:58:57.736969 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d45439-fa06-49dc-bb19-5ff57a3b08aa admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:58:57.747634 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d45439-fa06-49dc-bb19-5ff57a3b08aa admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:58:57.752334 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db996a14-051d-4ce4-a97c-3f9e8e25c45f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:58:57.752996 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db996a14-051d-4ce4-a97c-3f9e8e25c45f admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:57.753281 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db996a14-051d-4ce4-a97c-3f9e8e25c45f admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:57.753555 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db996a14-051d-4ce4-a97c-3f9e8e25c45f admin admin] PATH_INFO: `/v3/projects/49dfc44f75904964947f5382fe02fded/users/be47b7e8b99f4f7abda87a87c85100c2/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:57.769562 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db996a14-051d-4ce4-a97c-3f9e8e25c45f admin admin] RBAC: Authorizing `identity:create_grant(project_id=49dfc44f75904964947f5382fe02fded, user_id=be47b7e8b99f4f7abda87a87c85100c2, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:58:57.772253 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db996a14-051d-4ce4-a97c-3f9e8e25c45f admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:58:57.785129 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1356/2723] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:58:57 2022] PUT /identity/v3/projects/49dfc44f75904964947f5382fe02fded/users/be47b7e8b99f4f7abda87a87c85100c2/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:58:57.795719 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e22567d-a863-4b65-ac92-7b02b8f9e698 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:57.796003 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e22567d-a863-4b65-ac92-7b02b8f9e698 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:57.796273 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e22567d-a863-4b65-ac92-7b02b8f9e698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:57.824883 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-3e22567d-a863-4b65-ac92-7b02b8f9e698 None None] MFA Rules not processed for user `be47b7e8b99f4f7abda87a87c85100c2`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:58:57.826060 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-3e22567d-a863-4b65-ac92-7b02b8f9e698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:58:57.889370 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1368/2724] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:58:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:58:57.894003 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db996a14-051d-4ce4-a97c-3f9e8e25c45f admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:58:57.953867 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db996a14-051d-4ce4-a97c-3f9e8e25c45f admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:58:57.958968 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49420074-1351-4ee8-beb9-a11181b6e8b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:58:57.959600 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49420074-1351-4ee8-beb9-a11181b6e8b0 admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:57.959883 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49420074-1351-4ee8-beb9-a11181b6e8b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:57.960155 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49420074-1351-4ee8-beb9-a11181b6e8b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:57.960787 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49420074-1351-4ee8-beb9-a11181b6e8b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:58:57.962661 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49420074-1351-4ee8-beb9-a11181b6e8b0 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:58:57.989606 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1357/2725] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:58:57 2022] POST /identity/v3/projects => generated 385 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:58:57.993820 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e22567d-a863-4b65-ac92-7b02b8f9e698 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:58:58.004319 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e22567d-a863-4b65-ac92-7b02b8f9e698 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:58:58.048554 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:58:58.061449 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34ce786-6fb2-4c41-8a70-b6fadab39b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:58:58.062062 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ce786-6fb2-4c41-8a70-b6fadab39b9f admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:58.062342 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ce786-6fb2-4c41-8a70-b6fadab39b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:58.062616 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34ce786-6fb2-4c41-8a70-b6fadab39b9f admin admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:58.063256 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34ce786-6fb2-4c41-8a70-b6fadab39b9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:58:58.065272 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34ce786-6fb2-4c41-8a70-b6fadab39b9f admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:58:58.088342 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1369/2726] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:58:57 2022] POST /identity/v3/users => generated 299 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 10:58:58.092520 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49420074-1351-4ee8-beb9-a11181b6e8b0 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:58:58.102760 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49420074-1351-4ee8-beb9-a11181b6e8b0 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:58:58.107334 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2928ba-e77f-4c8b-b5af-65dd906e3791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:58:58.107945 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2928ba-e77f-4c8b-b5af-65dd906e3791 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:58.108245 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2928ba-e77f-4c8b-b5af-65dd906e3791 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:58.108514 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2928ba-e77f-4c8b-b5af-65dd906e3791 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:58.109142 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2928ba-e77f-4c8b-b5af-65dd906e3791 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:58:58.110971 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2928ba-e77f-4c8b-b5af-65dd906e3791 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:58:58.121642 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1358/2727] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:58:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:58:58.125836 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34ce786-6fb2-4c41-8a70-b6fadab39b9f admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:58:58.136481 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34ce786-6fb2-4c41-8a70-b6fadab39b9f admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:58:58.141240 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6818bda0-5c60-4035-92e6-722a377d18fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:58:58.141923 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6818bda0-5c60-4035-92e6-722a377d18fd admin admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:58.142203 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6818bda0-5c60-4035-92e6-722a377d18fd admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:58.142475 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6818bda0-5c60-4035-92e6-722a377d18fd admin admin] PATH_INFO: `/v3/projects/9d0daed0128d4c78991e42eb6e14c0b8/users/d396a33c7bdc49bf8f7e1e02a38830d3/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:58.157996 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6818bda0-5c60-4035-92e6-722a377d18fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d0daed0128d4c78991e42eb6e14c0b8, user_id=d396a33c7bdc49bf8f7e1e02a38830d3, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:58:58.160752 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6818bda0-5c60-4035-92e6-722a377d18fd admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:58:58.173147 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1370/2728] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:58:58 2022] PUT /identity/v3/projects/9d0daed0128d4c78991e42eb6e14c0b8/users/d396a33c7bdc49bf8f7e1e02a38830d3/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:58:58.177394 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2928ba-e77f-4c8b-b5af-65dd906e3791 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:58:58.187235 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2928ba-e77f-4c8b-b5af-65dd906e3791 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:58:58.237871 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:58:58.244867 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c79ad9-2437-435d-950c-3c7e08817ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:58:58.245812 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c79ad9-2437-435d-950c-3c7e08817ac8 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:58.246098 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c79ad9-2437-435d-950c-3c7e08817ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:58.246422 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c79ad9-2437-435d-950c-3c7e08817ac8 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:58.247205 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c79ad9-2437-435d-950c-3c7e08817ac8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:58:58.249206 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c79ad9-2437-435d-950c-3c7e08817ac8 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:58:58.260097 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1359/2729] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:58:58 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:58:58.264206 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6818bda0-5c60-4035-92e6-722a377d18fd admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:58:58.274804 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6818bda0-5c60-4035-92e6-722a377d18fd admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:58:58.279353 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6bf092-8efc-439b-8064-83fd6f5acff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:58:58.280049 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6bf092-8efc-439b-8064-83fd6f5acff0 admin admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:58.280331 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6bf092-8efc-439b-8064-83fd6f5acff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:58.280607 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6bf092-8efc-439b-8064-83fd6f5acff0 admin admin] PATH_INFO: `/v3/projects/9d0daed0128d4c78991e42eb6e14c0b8/users/d396a33c7bdc49bf8f7e1e02a38830d3/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:58.282516 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6bf092-8efc-439b-8064-83fd6f5acff0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d0daed0128d4c78991e42eb6e14c0b8, user_id=d396a33c7bdc49bf8f7e1e02a38830d3, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:58:58.285184 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6bf092-8efc-439b-8064-83fd6f5acff0 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:58:58.296204 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1371/2730] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:58:58 2022] PUT /identity/v3/projects/9d0daed0128d4c78991e42eb6e14c0b8/users/d396a33c7bdc49bf8f7e1e02a38830d3/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:58:58.308274 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cecabb-f504-4487-ae01-9f585b81a491 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:58.308758 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cecabb-f504-4487-ae01-9f585b81a491 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:58.308856 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cecabb-f504-4487-ae01-9f585b81a491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:58.340743 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-85cecabb-f504-4487-ae01-9f585b81a491 None None] MFA Rules not processed for user `d396a33c7bdc49bf8f7e1e02a38830d3`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:58:58.341869 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-85cecabb-f504-4487-ae01-9f585b81a491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:58:58.410708 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1360/2731] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:58:58 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:58:58.421390 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6bf092-8efc-439b-8064-83fd6f5acff0 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:58:58.423164 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-1c6bf092-8efc-439b-8064-83fd6f5acff0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:58:58.479603 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6bf092-8efc-439b-8064-83fd6f5acff0 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:58:58.483167 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206896ce-f949-4d1a-a239-0101c5a987be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:58:58.483795 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206896ce-f949-4d1a-a239-0101c5a987be service cinder] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:58.484073 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206896ce-f949-4d1a-a239-0101c5a987be service cinder] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:58.484341 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206896ce-f949-4d1a-a239-0101c5a987be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:58.494028 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206896ce-f949-4d1a-a239-0101c5a987be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:58:58.495756 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206896ce-f949-4d1a-a239-0101c5a987be service cinder] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:58:58.505771 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1372/2732] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:58:58 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:58:59.058273 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cecabb-f504-4487-ae01-9f585b81a491 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:58:59.120242 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cecabb-f504-4487-ae01-9f585b81a491 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:58:59.124702 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e0cbd2-da70-4ef8-b051-7b0017346a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:58:59.125641 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e0cbd2-da70-4ef8-b051-7b0017346a33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:59.125930 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e0cbd2-da70-4ef8-b051-7b0017346a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:59.126208 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e0cbd2-da70-4ef8-b051-7b0017346a33 admin admin] PATH_INFO: `/v3/users/be47b7e8b99f4f7abda87a87c85100c2` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:59.127326 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e0cbd2-da70-4ef8-b051-7b0017346a33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be47b7e8b99f4f7abda87a87c85100c2)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:58:59.129306 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e0cbd2-da70-4ef8-b051-7b0017346a33 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:58:59.175640 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-08e0cbd2-da70-4ef8-b051-7b0017346a33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be47b7e8b99f4f7abda87a87c85100c2'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:59.183593 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-08e0cbd2-da70-4ef8-b051-7b0017346a33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be47b7e8b99f4f7abda87a87c85100c2'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:59.184216 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-08e0cbd2-da70-4ef8-b051-7b0017346a33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be47b7e8b99f4f7abda87a87c85100c2'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:59.184826 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-08e0cbd2-da70-4ef8-b051-7b0017346a33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be47b7e8b99f4f7abda87a87c85100c2'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:59.191833 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-08e0cbd2-da70-4ef8-b051-7b0017346a33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be47b7e8b99f4f7abda87a87c85100c2'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:59.204431 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:58:59.210989 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1361/2733] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:58:59 2022] DELETE /identity/v3/users/be47b7e8b99f4f7abda87a87c85100c2 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:58:59.215486 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50908dd-f8e3-4aec-bf8e-9e084ae5e7ae None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:59.215780 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50908dd-f8e3-4aec-bf8e-9e084ae5e7ae None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:59.216089 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50908dd-f8e3-4aec-bf8e-9e084ae5e7ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:59.235932 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-a50908dd-f8e3-4aec-bf8e-9e084ae5e7ae None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:58:59.237124 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-a50908dd-f8e3-4aec-bf8e-9e084ae5e7ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:58:59.295473 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1373/2734] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:58:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:58:59.309469 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e0cbd2-da70-4ef8-b051-7b0017346a33 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:58:59.311627 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-08e0cbd2-da70-4ef8-b051-7b0017346a33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:58:59.373316 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e0cbd2-da70-4ef8-b051-7b0017346a33 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:58:59.376984 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be0ca9f-b560-4ec5-a3c5-6f30266c106b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:58:59.377580 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be0ca9f-b560-4ec5-a3c5-6f30266c106b service neutron] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:59.377855 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be0ca9f-b560-4ec5-a3c5-6f30266c106b service neutron] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:59.378123 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be0ca9f-b560-4ec5-a3c5-6f30266c106b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:59.388187 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be0ca9f-b560-4ec5-a3c5-6f30266c106b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:58:59.389586 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be0ca9f-b560-4ec5-a3c5-6f30266c106b service neutron] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:58:59.400025 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1362/2735] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:58:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:58:59.671738 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50908dd-f8e3-4aec-bf8e-9e084ae5e7ae None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:58:59.673185 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-a50908dd-f8e3-4aec-bf8e-9e084ae5e7ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:58:59.686921 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50908dd-f8e3-4aec-bf8e-9e084ae5e7ae None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:58:59.691535 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4cde18-2074-4d0f-b1a1-a3cee24a2062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:58:59.692683 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4cde18-2074-4d0f-b1a1-a3cee24a2062 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:59.692968 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4cde18-2074-4d0f-b1a1-a3cee24a2062 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:59.693229 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4cde18-2074-4d0f-b1a1-a3cee24a2062 admin admin] PATH_INFO: `/v3/projects/49dfc44f75904964947f5382fe02fded` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:59.694346 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4cde18-2074-4d0f-b1a1-a3cee24a2062 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49dfc44f75904964947f5382fe02fded)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:58:59.696334 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4cde18-2074-4d0f-b1a1-a3cee24a2062 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:58:59.736700 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:58:59.745365 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-cf4cde18-2074-4d0f-b1a1-a3cee24a2062 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49dfc44f75904964947f5382fe02fded', 'request_id': 'req-cf4cde18-2074-4d0f-b1a1-a3cee24a2062'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:59.752170 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-cf4cde18-2074-4d0f-b1a1-a3cee24a2062 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49dfc44f75904964947f5382fe02fded', 'request_id': 'req-cf4cde18-2074-4d0f-b1a1-a3cee24a2062'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:59.759178 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-cf4cde18-2074-4d0f-b1a1-a3cee24a2062 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49dfc44f75904964947f5382fe02fded', 'request_id': 'req-cf4cde18-2074-4d0f-b1a1-a3cee24a2062'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:59.772972 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-cf4cde18-2074-4d0f-b1a1-a3cee24a2062 admin admin] The token cache is being invalidate because project 49dfc44f75904964947f5382fe02fded was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:58:59.773265 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-cf4cde18-2074-4d0f-b1a1-a3cee24a2062 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:59.773899 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-cf4cde18-2074-4d0f-b1a1-a3cee24a2062 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:59.775444 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1374/2736] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:58:59 2022] DELETE /identity/v3/projects/49dfc44f75904964947f5382fe02fded => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:58:59.779499 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be0ca9f-b560-4ec5-a3c5-6f30266c106b service neutron] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:58:59.789884 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be0ca9f-b560-4ec5-a3c5-6f30266c106b service neutron] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:58:59.848996 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d68597-5359-4b18-965e-60efc99f2e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:58:59.849626 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d68597-5359-4b18-965e-60efc99f2e93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:58:59.849907 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d68597-5359-4b18-965e-60efc99f2e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:58:59.850181 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d68597-5359-4b18-965e-60efc99f2e93 admin admin] PATH_INFO: `/v3/users/d396a33c7bdc49bf8f7e1e02a38830d3` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:58:59.851307 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d68597-5359-4b18-965e-60efc99f2e93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d396a33c7bdc49bf8f7e1e02a38830d3)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:58:59.853253 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d68597-5359-4b18-965e-60efc99f2e93 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:58:59.901349 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-a8d68597-5359-4b18-965e-60efc99f2e93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd396a33c7bdc49bf8f7e1e02a38830d3'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:59.908799 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-a8d68597-5359-4b18-965e-60efc99f2e93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd396a33c7bdc49bf8f7e1e02a38830d3'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:59.909587 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-a8d68597-5359-4b18-965e-60efc99f2e93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd396a33c7bdc49bf8f7e1e02a38830d3'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:59.910192 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-a8d68597-5359-4b18-965e-60efc99f2e93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd396a33c7bdc49bf8f7e1e02a38830d3'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:59.917557 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-a8d68597-5359-4b18-965e-60efc99f2e93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd396a33c7bdc49bf8f7e1e02a38830d3'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:58:59.937077 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1363/2737] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:58:59 2022] DELETE /identity/v3/users/d396a33c7bdc49bf8f7e1e02a38830d3 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:59:00.212904 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4cde18-2074-4d0f-b1a1-a3cee24a2062 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:59:00.214428 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-cf4cde18-2074-4d0f-b1a1-a3cee24a2062 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:59:00.228967 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4cde18-2074-4d0f-b1a1-a3cee24a2062 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:59:00.286905 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1994782-6300-4e15-97da-fa7ef9096f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:59:00.287582 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1994782-6300-4e15-97da-fa7ef9096f1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:00.287866 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1994782-6300-4e15-97da-fa7ef9096f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:00.288140 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1994782-6300-4e15-97da-fa7ef9096f1d admin admin] PATH_INFO: `/v3/projects/9d0daed0128d4c78991e42eb6e14c0b8` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:00.289265 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1994782-6300-4e15-97da-fa7ef9096f1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d0daed0128d4c78991e42eb6e14c0b8)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:59:00.291176 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1994782-6300-4e15-97da-fa7ef9096f1d admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:59:00.337343 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-c1994782-6300-4e15-97da-fa7ef9096f1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d0daed0128d4c78991e42eb6e14c0b8', 'request_id': 'req-c1994782-6300-4e15-97da-fa7ef9096f1d'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:59:00.343658 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-c1994782-6300-4e15-97da-fa7ef9096f1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d0daed0128d4c78991e42eb6e14c0b8', 'request_id': 'req-c1994782-6300-4e15-97da-fa7ef9096f1d'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:59:00.349859 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-c1994782-6300-4e15-97da-fa7ef9096f1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d0daed0128d4c78991e42eb6e14c0b8', 'request_id': 'req-c1994782-6300-4e15-97da-fa7ef9096f1d'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:59:00.361894 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-c1994782-6300-4e15-97da-fa7ef9096f1d admin admin] The token cache is being invalidate because project 9d0daed0128d4c78991e42eb6e14c0b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:59:00.362189 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-c1994782-6300-4e15-97da-fa7ef9096f1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:59:00.362830 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-c1994782-6300-4e15-97da-fa7ef9096f1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:59:00.365098 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1375/2738] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:59:00 2022] DELETE /identity/v3/projects/9d0daed0128d4c78991e42eb6e14c0b8 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:59:00.369926 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ccfe0-af13-4b00-975a-a9d7470953c7 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:00.370420 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ccfe0-af13-4b00-975a-a9d7470953c7 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:00.370530 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ccfe0-af13-4b00-975a-a9d7470953c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:00.389851 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-7d0ccfe0-af13-4b00-975a-a9d7470953c7 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:59:00.391048 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-7d0ccfe0-af13-4b00-975a-a9d7470953c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:59:00.403769 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:59:00.452032 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1364/2739] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:59:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 10:59:00.459835 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b939a-1f57-425b-9fc8-89f5126b47b6 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:00.460178 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b939a-1f57-425b-9fc8-89f5126b47b6 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:00.460460 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b939a-1f57-425b-9fc8-89f5126b47b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:00.481324 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-a16b939a-1f57-425b-9fc8-89f5126b47b6 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:59:00.482591 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-a16b939a-1f57-425b-9fc8-89f5126b47b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:59:00.491757 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1376/2740] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:59:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:59:00.495872 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ccfe0-af13-4b00-975a-a9d7470953c7 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:59:00.505577 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ccfe0-af13-4b00-975a-a9d7470953c7 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:59:00.510168 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47102748-15fc-4b38-9012-563d9a7ce267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:59:00.510761 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47102748-15fc-4b38-9012-563d9a7ce267 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:00.511045 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47102748-15fc-4b38-9012-563d9a7ce267 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:00.511319 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47102748-15fc-4b38-9012-563d9a7ce267 admin admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:00.511997 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47102748-15fc-4b38-9012-563d9a7ce267 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:59:00.512299 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47102748-15fc-4b38-9012-563d9a7ce267 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:59:00.513898 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47102748-15fc-4b38-9012-563d9a7ce267 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:59:00.527118 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1365/2741] 10.222.0.15 () {58 vars in 1153 bytes} [Sun Sep 11 10:59:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:59:00.529742 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16b939a-1f57-425b-9fc8-89f5126b47b6 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:59:00.540932 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16b939a-1f57-425b-9fc8-89f5126b47b6 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:59:00.546088 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2fd6d9-efec-4a17-9d4c-c90a4a1d3b94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:59:00.546717 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2fd6d9-efec-4a17-9d4c-c90a4a1d3b94 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:00.546999 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2fd6d9-efec-4a17-9d4c-c90a4a1d3b94 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:00.547274 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2fd6d9-efec-4a17-9d4c-c90a4a1d3b94 admin admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:00.547947 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2fd6d9-efec-4a17-9d4c-c90a4a1d3b94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:59:00.549864 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2fd6d9-efec-4a17-9d4c-c90a4a1d3b94 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:59:00.577938 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1377/2742] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:59:00 2022] POST /identity/v3/projects => generated 369 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:59:00.582244 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47102748-15fc-4b38-9012-563d9a7ce267 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:59:00.593043 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47102748-15fc-4b38-9012-563d9a7ce267 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:59:00.649377 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e630ab8a-f1ae-4ec9-b82f-63a7d272ec2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:59:00.650051 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e630ab8a-f1ae-4ec9-b82f-63a7d272ec2a admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:00.650333 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e630ab8a-f1ae-4ec9-b82f-63a7d272ec2a admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:00.650605 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e630ab8a-f1ae-4ec9-b82f-63a7d272ec2a admin admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:00.651226 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e630ab8a-f1ae-4ec9-b82f-63a7d272ec2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:59:00.653201 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e630ab8a-f1ae-4ec9-b82f-63a7d272ec2a admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:59:00.675316 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1366/2743] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:59:00 2022] POST /identity/v3/users => generated 285 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:59:00.680250 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2fd6d9-efec-4a17-9d4c-c90a4a1d3b94 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:59:00.690440 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2fd6d9-efec-4a17-9d4c-c90a4a1d3b94 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:59:00.695840 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a0d747-f31a-46de-a761-4f770b596a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:59:00.696440 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a0d747-f31a-46de-a761-4f770b596a14 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:00.696726 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a0d747-f31a-46de-a761-4f770b596a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:00.697007 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a0d747-f31a-46de-a761-4f770b596a14 admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:00.697634 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a0d747-f31a-46de-a761-4f770b596a14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:59:00.699225 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a0d747-f31a-46de-a761-4f770b596a14 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:59:00.709696 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1378/2744] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:59:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:59:00.713467 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e630ab8a-f1ae-4ec9-b82f-63a7d272ec2a admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:59:00.723858 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e630ab8a-f1ae-4ec9-b82f-63a7d272ec2a admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:59:00.728683 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdc0d5c-2de4-4d8c-8c01-17bc9c8404b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:59:00.729348 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc0d5c-2de4-4d8c-8c01-17bc9c8404b6 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:00.729678 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc0d5c-2de4-4d8c-8c01-17bc9c8404b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:00.729975 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc0d5c-2de4-4d8c-8c01-17bc9c8404b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:00.730692 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdc0d5c-2de4-4d8c-8c01-17bc9c8404b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:59:00.732320 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdc0d5c-2de4-4d8c-8c01-17bc9c8404b6 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:59:00.743786 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1367/2745] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:59:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:59:00.747822 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a0d747-f31a-46de-a761-4f770b596a14 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:59:00.758285 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a0d747-f31a-46de-a761-4f770b596a14 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:59:00.762894 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb52062-f37d-442a-a7b6-2b91ca92cb1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:59:00.763570 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb52062-f37d-442a-a7b6-2b91ca92cb1a admin admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:00.763846 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb52062-f37d-442a-a7b6-2b91ca92cb1a admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:00.764116 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb52062-f37d-442a-a7b6-2b91ca92cb1a admin admin] PATH_INFO: `/v3/projects/8daea864b58c4269b9c915a474e1337a/users/dc5d4549437d44809156e25a9fbb85e9/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:00.780026 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb52062-f37d-442a-a7b6-2b91ca92cb1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8daea864b58c4269b9c915a474e1337a, user_id=dc5d4549437d44809156e25a9fbb85e9, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:59:00.782674 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb52062-f37d-442a-a7b6-2b91ca92cb1a admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:59:00.784821 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:59:00.796659 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1379/2746] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:59:00 2022] PUT /identity/v3/projects/8daea864b58c4269b9c915a474e1337a/users/dc5d4549437d44809156e25a9fbb85e9/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:59:00.802181 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb1a92-3259-4ad9-b41e-782406475987 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:00.802468 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb1a92-3259-4ad9-b41e-782406475987 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:00.802740 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb1a92-3259-4ad9-b41e-782406475987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:00.822266 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-94fb1a92-3259-4ad9-b41e-782406475987 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:59:00.823497 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-94fb1a92-3259-4ad9-b41e-782406475987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:59:00.884097 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1368/2747] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:59:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:59:00.893545 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb52062-f37d-442a-a7b6-2b91ca92cb1a admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:59:00.895908 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-8fb52062-f37d-442a-a7b6-2b91ca92cb1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:59:00.951601 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb52062-f37d-442a-a7b6-2b91ca92cb1a admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:59:00.955255 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de57d77d-e662-42fc-8a05-295ff5e3034d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:59:00.955920 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57d77d-e662-42fc-8a05-295ff5e3034d service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:00.956203 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57d77d-e662-42fc-8a05-295ff5e3034d service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:00.956507 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57d77d-e662-42fc-8a05-295ff5e3034d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:00.966510 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de57d77d-e662-42fc-8a05-295ff5e3034d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:59:00.967904 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de57d77d-e662-42fc-8a05-295ff5e3034d service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:59:00.977711 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1380/2748] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:59:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:59:01.597080 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9210e703-aee9-4302-a422-99543ce2a369 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:01.597603 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9210e703-aee9-4302-a422-99543ce2a369 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:01.597603 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9210e703-aee9-4302-a422-99543ce2a369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:01.626246 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-9210e703-aee9-4302-a422-99543ce2a369 None None] MFA Rules not processed for user `dc5d4549437d44809156e25a9fbb85e9`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:59:01.627542 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-9210e703-aee9-4302-a422-99543ce2a369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:59:01.641807 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:59:01.698986 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1369/2749] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:59:01 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:59:01.703820 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de57d77d-e662-42fc-8a05-295ff5e3034d service neutron] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:59:01.714052 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de57d77d-e662-42fc-8a05-295ff5e3034d service neutron] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:59:01.718621 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef37e4dc-9fd4-4713-91db-991aadc96281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:59:01.719223 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef37e4dc-9fd4-4713-91db-991aadc96281 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:01.719540 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef37e4dc-9fd4-4713-91db-991aadc96281 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:01.719817 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef37e4dc-9fd4-4713-91db-991aadc96281 admin admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:01.720473 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef37e4dc-9fd4-4713-91db-991aadc96281 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:59:01.722726 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef37e4dc-9fd4-4713-91db-991aadc96281 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:59:01.750694 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1381/2750] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:59:01 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:59:01.755513 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9210e703-aee9-4302-a422-99543ce2a369 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:59:01.766224 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9210e703-aee9-4302-a422-99543ce2a369 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:59:01.824737 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80692b6-8b5f-41b4-90d3-803a6a0c9f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:59:01.825343 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80692b6-8b5f-41b4-90d3-803a6a0c9f94 admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:01.825623 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80692b6-8b5f-41b4-90d3-803a6a0c9f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:01.825889 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80692b6-8b5f-41b4-90d3-803a6a0c9f94 admin admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:01.826626 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80692b6-8b5f-41b4-90d3-803a6a0c9f94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:59:01.828941 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80692b6-8b5f-41b4-90d3-803a6a0c9f94 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:59:01.850817 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1370/2751] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:59:01 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 10:59:01.854897 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef37e4dc-9fd4-4713-91db-991aadc96281 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:59:01.865413 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef37e4dc-9fd4-4713-91db-991aadc96281 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:59:01.870108 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0e83af-ab99-4b7d-8ef7-7d404cca6f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:59:01.870713 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0e83af-ab99-4b7d-8ef7-7d404cca6f44 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:01.870992 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0e83af-ab99-4b7d-8ef7-7d404cca6f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:01.871262 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0e83af-ab99-4b7d-8ef7-7d404cca6f44 admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:01.871931 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0e83af-ab99-4b7d-8ef7-7d404cca6f44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:59:01.873572 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0e83af-ab99-4b7d-8ef7-7d404cca6f44 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:59:01.884844 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1382/2752] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:59:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:59:01.889033 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80692b6-8b5f-41b4-90d3-803a6a0c9f94 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:59:01.899894 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80692b6-8b5f-41b4-90d3-803a6a0c9f94 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:59:01.904557 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a962ba2f-2552-44c6-bc6a-9a8d2ef95c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:59:01.905232 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a962ba2f-2552-44c6-bc6a-9a8d2ef95c5e admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:01.905518 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a962ba2f-2552-44c6-bc6a-9a8d2ef95c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:01.905857 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a962ba2f-2552-44c6-bc6a-9a8d2ef95c5e admin admin] PATH_INFO: `/v3/projects/0e28f30f382a4fb29b48ee708f8f44a9/users/219e308abbe04445ab6c183677684473/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:01.920697 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a962ba2f-2552-44c6-bc6a-9a8d2ef95c5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e28f30f382a4fb29b48ee708f8f44a9, user_id=219e308abbe04445ab6c183677684473, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:59:01.923347 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a962ba2f-2552-44c6-bc6a-9a8d2ef95c5e admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:59:01.936714 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1371/2753] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:59:01 2022] PUT /identity/v3/projects/0e28f30f382a4fb29b48ee708f8f44a9/users/219e308abbe04445ab6c183677684473/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 10:59:01.940669 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0e83af-ab99-4b7d-8ef7-7d404cca6f44 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:59:01.951036 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0e83af-ab99-4b7d-8ef7-7d404cca6f44 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:59:02.006785 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eca0f93-2129-4266-8ef4-3b4e410e3b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:59:02.007376 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eca0f93-2129-4266-8ef4-3b4e410e3b40 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:02.007689 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eca0f93-2129-4266-8ef4-3b4e410e3b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:02.007964 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eca0f93-2129-4266-8ef4-3b4e410e3b40 admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:02.008597 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eca0f93-2129-4266-8ef4-3b4e410e3b40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:59:02.010188 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eca0f93-2129-4266-8ef4-3b4e410e3b40 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:59:02.019779 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1383/2754] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:59:01 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:59:02.024609 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a962ba2f-2552-44c6-bc6a-9a8d2ef95c5e admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:59:02.036145 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a962ba2f-2552-44c6-bc6a-9a8d2ef95c5e admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:59:02.040932 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f0bb80-f403-4902-a94a-04732f559cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:59:02.041598 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0bb80-f403-4902-a94a-04732f559cfe admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:02.041872 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0bb80-f403-4902-a94a-04732f559cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:02.042136 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0bb80-f403-4902-a94a-04732f559cfe admin admin] PATH_INFO: `/v3/projects/0e28f30f382a4fb29b48ee708f8f44a9/users/219e308abbe04445ab6c183677684473/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:02.044031 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f0bb80-f403-4902-a94a-04732f559cfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e28f30f382a4fb29b48ee708f8f44a9, user_id=219e308abbe04445ab6c183677684473, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:59:02.046923 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f0bb80-f403-4902-a94a-04732f559cfe admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:59:02.057938 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1372/2755] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:59:02 2022] PUT /identity/v3/projects/0e28f30f382a4fb29b48ee708f8f44a9/users/219e308abbe04445ab6c183677684473/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:59:02.679398 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812736fe-b0e8-4a79-9acb-adf90ac321a2 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:02.679719 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812736fe-b0e8-4a79-9acb-adf90ac321a2 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:02.679994 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812736fe-b0e8-4a79-9acb-adf90ac321a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:02.711365 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-812736fe-b0e8-4a79-9acb-adf90ac321a2 None None] MFA Rules not processed for user `219e308abbe04445ab6c183677684473`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:59:02.712606 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-812736fe-b0e8-4a79-9acb-adf90ac321a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:59:02.765316 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:59:02.789001 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1384/2756] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:59:02 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 10:59:02.800393 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f0bb80-f403-4902-a94a-04732f559cfe admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:59:02.802880 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-55f0bb80-f403-4902-a94a-04732f559cfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:59:02.868520 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f0bb80-f403-4902-a94a-04732f559cfe admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:59:02.872054 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119e00d7-4752-4b1b-a80b-7b871874a33e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:59:02.872658 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119e00d7-4752-4b1b-a80b-7b871874a33e service cinder] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:02.872937 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119e00d7-4752-4b1b-a80b-7b871874a33e service cinder] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:02.873207 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119e00d7-4752-4b1b-a80b-7b871874a33e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:02.893522 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:59:02.931230 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119e00d7-4752-4b1b-a80b-7b871874a33e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:59:02.932688 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119e00d7-4752-4b1b-a80b-7b871874a33e service cinder] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:59:02.941457 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1373/2757] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:59:02 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:59:04.310450 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812736fe-b0e8-4a79-9acb-adf90ac321a2 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:59:04.311871 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-812736fe-b0e8-4a79-9acb-adf90ac321a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:59:04.324104 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812736fe-b0e8-4a79-9acb-adf90ac321a2 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:59:04.327748 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14638f0f-1706-4b04-9cb2-4a1a70738de1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:59:04.328347 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14638f0f-1706-4b04-9cb2-4a1a70738de1 service cinder] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:04.328687 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14638f0f-1706-4b04-9cb2-4a1a70738de1 service cinder] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:04.328959 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14638f0f-1706-4b04-9cb2-4a1a70738de1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:04.337885 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14638f0f-1706-4b04-9cb2-4a1a70738de1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:59:04.339345 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14638f0f-1706-4b04-9cb2-4a1a70738de1 service cinder] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:59:04.349430 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1385/2758] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:59:04 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:59:11.390372 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119e00d7-4752-4b1b-a80b-7b871874a33e service cinder] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:59:11.445794 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119e00d7-4752-4b1b-a80b-7b871874a33e service cinder] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:59:11.449312 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55252d1a-52c9-4846-b75a-df13130a09c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:59:11.449916 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55252d1a-52c9-4846-b75a-df13130a09c5 service neutron] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:11.450194 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55252d1a-52c9-4846-b75a-df13130a09c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:11.450466 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55252d1a-52c9-4846-b75a-df13130a09c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:11.452212 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-55252d1a-52c9-4846-b75a-df13130a09c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:59:11.513958 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55252d1a-52c9-4846-b75a-df13130a09c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:59:11.515373 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55252d1a-52c9-4846-b75a-df13130a09c5 service neutron] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:59:11.525155 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1374/2759] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:59:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:59:14.344716 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=13 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:59:14.661387 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14638f0f-1706-4b04-9cb2-4a1a70738de1 service cinder] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:59:14.663792 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-14638f0f-1706-4b04-9cb2-4a1a70738de1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:59:14.685469 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14638f0f-1706-4b04-9cb2-4a1a70738de1 service cinder] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:59:14.690440 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d7af7c-f8ed-4b27-977a-dd046ca01c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:59:14.691157 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d7af7c-f8ed-4b27-977a-dd046ca01c44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:14.691556 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d7af7c-f8ed-4b27-977a-dd046ca01c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:14.692691 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d7af7c-f8ed-4b27-977a-dd046ca01c44 admin admin] PATH_INFO: `/v3/users/2238a62cbf1846999b31ea9434b67743` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:14.694540 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d7af7c-f8ed-4b27-977a-dd046ca01c44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2238a62cbf1846999b31ea9434b67743)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:59:14.696741 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d7af7c-f8ed-4b27-977a-dd046ca01c44 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:59:14.747952 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-31d7af7c-f8ed-4b27-977a-dd046ca01c44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2238a62cbf1846999b31ea9434b67743'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:59:14.756725 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-31d7af7c-f8ed-4b27-977a-dd046ca01c44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2238a62cbf1846999b31ea9434b67743'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:59:14.757662 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-31d7af7c-f8ed-4b27-977a-dd046ca01c44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2238a62cbf1846999b31ea9434b67743'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:59:14.758264 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-31d7af7c-f8ed-4b27-977a-dd046ca01c44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2238a62cbf1846999b31ea9434b67743'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:59:14.771083 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-31d7af7c-f8ed-4b27-977a-dd046ca01c44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2238a62cbf1846999b31ea9434b67743'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:59:14.786496 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1386/2760] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:59:14 2022] DELETE /identity/v3/users/2238a62cbf1846999b31ea9434b67743 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:59:14.962052 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55252d1a-52c9-4846-b75a-df13130a09c5 service neutron] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:59:14.973321 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55252d1a-52c9-4846-b75a-df13130a09c5 service neutron] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:59:15.035218 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09c729e-a2d6-4c96-b744-913a861d3845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:59:15.035877 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09c729e-a2d6-4c96-b744-913a861d3845 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:15.036174 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09c729e-a2d6-4c96-b744-913a861d3845 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:15.036525 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09c729e-a2d6-4c96-b744-913a861d3845 admin admin] PATH_INFO: `/v3/projects/d0f44d45f87c4b28a1672c48fd9613ca` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:15.037995 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09c729e-a2d6-4c96-b744-913a861d3845 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0f44d45f87c4b28a1672c48fd9613ca)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:59:15.040070 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09c729e-a2d6-4c96-b744-913a861d3845 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:59:15.059722 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=100 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:59:15.091877 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-f09c729e-a2d6-4c96-b744-913a861d3845 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0f44d45f87c4b28a1672c48fd9613ca', 'request_id': 'req-f09c729e-a2d6-4c96-b744-913a861d3845'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:59:15.098708 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-f09c729e-a2d6-4c96-b744-913a861d3845 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0f44d45f87c4b28a1672c48fd9613ca', 'request_id': 'req-f09c729e-a2d6-4c96-b744-913a861d3845'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:59:15.107329 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-f09c729e-a2d6-4c96-b744-913a861d3845 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0f44d45f87c4b28a1672c48fd9613ca', 'request_id': 'req-f09c729e-a2d6-4c96-b744-913a861d3845'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:59:15.120099 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-f09c729e-a2d6-4c96-b744-913a861d3845 admin admin] The token cache is being invalidate because project d0f44d45f87c4b28a1672c48fd9613ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90653) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:59:15.120390 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-f09c729e-a2d6-4c96-b744-913a861d3845 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:59:15.121029 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-f09c729e-a2d6-4c96-b744-913a861d3845 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:59:15.122722 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1375/2761] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:59:14 2022] DELETE /identity/v3/projects/d0f44d45f87c4b28a1672c48fd9613ca => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:59:15.127939 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeed2f6-1bfe-47ed-828a-4e78725cd90d None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:15.128230 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeed2f6-1bfe-47ed-828a-4e78725cd90d None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:15.128506 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeed2f6-1bfe-47ed-828a-4e78725cd90d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:15.149155 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-7aeed2f6-1bfe-47ed-828a-4e78725cd90d None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:59:15.150401 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-7aeed2f6-1bfe-47ed-828a-4e78725cd90d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:59:15.211570 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1387/2762] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:59:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:59:15.219356 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73afc471-210a-4d5a-99bc-a128503e6304 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:15.219672 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73afc471-210a-4d5a-99bc-a128503e6304 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:15.219948 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73afc471-210a-4d5a-99bc-a128503e6304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:15.241602 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-73afc471-210a-4d5a-99bc-a128503e6304 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:59:15.242737 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-73afc471-210a-4d5a-99bc-a128503e6304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:59:15.250837 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1376/2763] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:59:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 10:59:15.255386 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeed2f6-1bfe-47ed-828a-4e78725cd90d None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:59:15.266800 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeed2f6-1bfe-47ed-828a-4e78725cd90d None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:59:15.271555 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5430ee-0139-4473-9880-2ec579ba8d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:59:15.272239 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5430ee-0139-4473-9880-2ec579ba8d91 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:15.272564 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5430ee-0139-4473-9880-2ec579ba8d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:15.272769 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5430ee-0139-4473-9880-2ec579ba8d91 admin admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:15.273445 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5430ee-0139-4473-9880-2ec579ba8d91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:59:15.273753 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5430ee-0139-4473-9880-2ec579ba8d91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:59:15.275356 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5430ee-0139-4473-9880-2ec579ba8d91 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:59:15.291641 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1388/2764] 10.222.0.15 () {58 vars in 1153 bytes} [Sun Sep 11 10:59:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:59:15.295913 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73afc471-210a-4d5a-99bc-a128503e6304 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:59:15.310445 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73afc471-210a-4d5a-99bc-a128503e6304 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:59:15.316347 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45554c58-61ac-4f2b-b15b-dbd95eea72a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:59:15.317048 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45554c58-61ac-4f2b-b15b-dbd95eea72a8 admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:15.317265 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45554c58-61ac-4f2b-b15b-dbd95eea72a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:15.317563 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45554c58-61ac-4f2b-b15b-dbd95eea72a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:15.318196 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45554c58-61ac-4f2b-b15b-dbd95eea72a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:59:15.320370 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45554c58-61ac-4f2b-b15b-dbd95eea72a8 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:59:15.349577 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1377/2765] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:59:15 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:59:15.354270 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5430ee-0139-4473-9880-2ec579ba8d91 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:59:15.364667 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5430ee-0139-4473-9880-2ec579ba8d91 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:59:15.430381 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5ffe82-64e8-4371-8516-f545475a759a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:59:15.431291 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5ffe82-64e8-4371-8516-f545475a759a admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:15.431291 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5ffe82-64e8-4371-8516-f545475a759a admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:15.431629 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5ffe82-64e8-4371-8516-f545475a759a admin admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:15.432213 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5ffe82-64e8-4371-8516-f545475a759a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:59:15.434164 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5ffe82-64e8-4371-8516-f545475a759a admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:59:15.442780 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:59:15.458744 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1389/2766] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:59:15 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 10:59:15.463500 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45554c58-61ac-4f2b-b15b-dbd95eea72a8 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:59:15.473787 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45554c58-61ac-4f2b-b15b-dbd95eea72a8 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:59:15.478443 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6289b6ea-7e6d-43d7-92a7-584bb3f23496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:59:15.479180 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289b6ea-7e6d-43d7-92a7-584bb3f23496 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:15.479314 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289b6ea-7e6d-43d7-92a7-584bb3f23496 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:15.479607 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289b6ea-7e6d-43d7-92a7-584bb3f23496 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:15.480248 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6289b6ea-7e6d-43d7-92a7-584bb3f23496 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:59:15.481830 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6289b6ea-7e6d-43d7-92a7-584bb3f23496 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:59:15.492184 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1378/2767] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:59:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 10:59:15.496063 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5ffe82-64e8-4371-8516-f545475a759a admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:59:15.505801 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5ffe82-64e8-4371-8516-f545475a759a admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:59:15.511190 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ce28c3-3993-4d9f-b74e-b93bdbe5e445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:59:15.511827 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce28c3-3993-4d9f-b74e-b93bdbe5e445 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:15.512106 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce28c3-3993-4d9f-b74e-b93bdbe5e445 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:15.512377 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce28c3-3993-4d9f-b74e-b93bdbe5e445 admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:15.513036 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ce28c3-3993-4d9f-b74e-b93bdbe5e445 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:59:15.514635 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ce28c3-3993-4d9f-b74e-b93bdbe5e445 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:59:15.524949 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1390/2768] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:59:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 10:59:15.528881 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6289b6ea-7e6d-43d7-92a7-584bb3f23496 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:59:15.539064 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6289b6ea-7e6d-43d7-92a7-584bb3f23496 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:59:15.543731 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9ffc1a-37f0-4d9b-a7e2-1bb7b13bce7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:59:15.544389 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9ffc1a-37f0-4d9b-a7e2-1bb7b13bce7f admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:15.544770 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9ffc1a-37f0-4d9b-a7e2-1bb7b13bce7f admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:15.545108 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9ffc1a-37f0-4d9b-a7e2-1bb7b13bce7f admin admin] PATH_INFO: `/v3/projects/ecad09476bb3444da0ca3ef5342bb0ef/users/f071b2156180461aa0ee9c4c80acdc68/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:15.559818 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9ffc1a-37f0-4d9b-a7e2-1bb7b13bce7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecad09476bb3444da0ca3ef5342bb0ef, user_id=f071b2156180461aa0ee9c4c80acdc68, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:59:15.562495 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9ffc1a-37f0-4d9b-a7e2-1bb7b13bce7f admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:59:15.575841 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1379/2769] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:59:15 2022] PUT /identity/v3/projects/ecad09476bb3444da0ca3ef5342bb0ef/users/f071b2156180461aa0ee9c4c80acdc68/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:59:15.582000 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2f62c-b98b-4264-a0ea-e4b68b810c60 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:15.582289 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2f62c-b98b-4264-a0ea-e4b68b810c60 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:15.582563 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2f62c-b98b-4264-a0ea-e4b68b810c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:15.601700 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-7bf2f62c-b98b-4264-a0ea-e4b68b810c60 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:59:15.602901 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-7bf2f62c-b98b-4264-a0ea-e4b68b810c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:59:15.660167 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1391/2770] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:59:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:59:15.670103 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9ffc1a-37f0-4d9b-a7e2-1bb7b13bce7f admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:59:15.672153 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-0a9ffc1a-37f0-4d9b-a7e2-1bb7b13bce7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:59:15.733063 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9ffc1a-37f0-4d9b-a7e2-1bb7b13bce7f admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:59:15.736998 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da702281-ab42-48b8-acbc-b48f05706675 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:59:15.737598 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da702281-ab42-48b8-acbc-b48f05706675 service neutron] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:15.737879 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da702281-ab42-48b8-acbc-b48f05706675 service neutron] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:15.738154 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da702281-ab42-48b8-acbc-b48f05706675 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:15.747985 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da702281-ab42-48b8-acbc-b48f05706675 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:59:15.749329 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da702281-ab42-48b8-acbc-b48f05706675 service neutron] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:59:15.759507 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1380/2771] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:59:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:59:17.801916 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf2f62c-b98b-4264-a0ea-e4b68b810c60 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:59:17.803647 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-7bf2f62c-b98b-4264-a0ea-e4b68b810c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:59:17.820618 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf2f62c-b98b-4264-a0ea-e4b68b810c60 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:59:17.825976 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e8505e-a3a2-4279-8e82-a441b34d9906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:59:17.826995 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8505e-a3a2-4279-8e82-a441b34d9906 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:17.827344 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8505e-a3a2-4279-8e82-a441b34d9906 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:17.827718 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8505e-a3a2-4279-8e82-a441b34d9906 admin admin] PATH_INFO: `/v3/users/dc5d4549437d44809156e25a9fbb85e9` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:17.830557 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e8505e-a3a2-4279-8e82-a441b34d9906 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc5d4549437d44809156e25a9fbb85e9)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:59:17.832657 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e8505e-a3a2-4279-8e82-a441b34d9906 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:59:17.884204 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-54e8505e-a3a2-4279-8e82-a441b34d9906 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc5d4549437d44809156e25a9fbb85e9'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:59:17.891628 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-54e8505e-a3a2-4279-8e82-a441b34d9906 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc5d4549437d44809156e25a9fbb85e9'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:59:17.892513 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-54e8505e-a3a2-4279-8e82-a441b34d9906 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc5d4549437d44809156e25a9fbb85e9'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:59:17.893127 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-54e8505e-a3a2-4279-8e82-a441b34d9906 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc5d4549437d44809156e25a9fbb85e9'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:59:17.906623 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-54e8505e-a3a2-4279-8e82-a441b34d9906 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc5d4549437d44809156e25a9fbb85e9'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:59:17.922822 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1392/2772] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:59:17 2022] DELETE /identity/v3/users/dc5d4549437d44809156e25a9fbb85e9 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:59:18.124365 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da702281-ab42-48b8-acbc-b48f05706675 service neutron] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:59:18.144699 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da702281-ab42-48b8-acbc-b48f05706675 service neutron] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:59:18.204267 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:59:18.243688 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8edad98-6106-4db3-95be-2f7f160a2bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:59:18.245357 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8edad98-6106-4db3-95be-2f7f160a2bc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:18.246911 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8edad98-6106-4db3-95be-2f7f160a2bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:18.247185 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8edad98-6106-4db3-95be-2f7f160a2bc0 admin admin] PATH_INFO: `/v3/projects/8daea864b58c4269b9c915a474e1337a` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:18.248610 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8edad98-6106-4db3-95be-2f7f160a2bc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8daea864b58c4269b9c915a474e1337a)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:59:18.252473 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8edad98-6106-4db3-95be-2f7f160a2bc0 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:59:18.349246 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-e8edad98-6106-4db3-95be-2f7f160a2bc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8daea864b58c4269b9c915a474e1337a', 'request_id': 'req-e8edad98-6106-4db3-95be-2f7f160a2bc0'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:59:18.369132 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-e8edad98-6106-4db3-95be-2f7f160a2bc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8daea864b58c4269b9c915a474e1337a', 'request_id': 'req-e8edad98-6106-4db3-95be-2f7f160a2bc0'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:59:18.383605 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-e8edad98-6106-4db3-95be-2f7f160a2bc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8daea864b58c4269b9c915a474e1337a', 'request_id': 'req-e8edad98-6106-4db3-95be-2f7f160a2bc0'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:59:18.400457 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-e8edad98-6106-4db3-95be-2f7f160a2bc0 admin admin] The token cache is being invalidate because project 8daea864b58c4269b9c915a474e1337a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90653) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:59:18.400675 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-e8edad98-6106-4db3-95be-2f7f160a2bc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:59:18.401993 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-e8edad98-6106-4db3-95be-2f7f160a2bc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:59:18.403576 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1381/2773] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:59:18 2022] DELETE /identity/v3/projects/8daea864b58c4269b9c915a474e1337a => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:59:18.409960 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e8505e-a3a2-4279-8e82-a441b34d9906 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:59:18.411553 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-54e8505e-a3a2-4279-8e82-a441b34d9906 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:59:18.430387 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e8505e-a3a2-4279-8e82-a441b34d9906 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:59:18.503248 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:INSERT=4,keystone:SELECT=86,keystone:DELETE=10 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:59:18.525322 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d1a933-9676-4389-8207-0141aeb08abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:59:18.526111 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d1a933-9676-4389-8207-0141aeb08abe admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:18.526488 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d1a933-9676-4389-8207-0141aeb08abe admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:18.527044 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d1a933-9676-4389-8207-0141aeb08abe admin admin] PATH_INFO: `/v3/users/219e308abbe04445ab6c183677684473` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:18.528784 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d1a933-9676-4389-8207-0141aeb08abe admin admin] RBAC: Authorizing `identity:delete_user(user_id=219e308abbe04445ab6c183677684473)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:59:18.530920 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d1a933-9676-4389-8207-0141aeb08abe admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:59:18.608535 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-e7d1a933-9676-4389-8207-0141aeb08abe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '219e308abbe04445ab6c183677684473'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:59:18.624522 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-e7d1a933-9676-4389-8207-0141aeb08abe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '219e308abbe04445ab6c183677684473'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:59:18.625950 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-e7d1a933-9676-4389-8207-0141aeb08abe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '219e308abbe04445ab6c183677684473'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:59:18.626961 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-e7d1a933-9676-4389-8207-0141aeb08abe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '219e308abbe04445ab6c183677684473'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:59:18.640932 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-e7d1a933-9676-4389-8207-0141aeb08abe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '219e308abbe04445ab6c183677684473'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:59:18.664552 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1393/2774] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:59:18 2022] DELETE /identity/v3/users/219e308abbe04445ab6c183677684473 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:59:18.895043 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8edad98-6106-4db3-95be-2f7f160a2bc0 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:59:18.896690 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-e8edad98-6106-4db3-95be-2f7f160a2bc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:59:18.916001 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8edad98-6106-4db3-95be-2f7f160a2bc0 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:59:19.015192 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3046f29e-6945-42ff-a332-20319fac9713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:59:19.020531 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3046f29e-6945-42ff-a332-20319fac9713 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:19.020531 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3046f29e-6945-42ff-a332-20319fac9713 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:19.020531 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3046f29e-6945-42ff-a332-20319fac9713 admin admin] PATH_INFO: `/v3/projects/0e28f30f382a4fb29b48ee708f8f44a9` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:19.020531 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3046f29e-6945-42ff-a332-20319fac9713 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e28f30f382a4fb29b48ee708f8f44a9)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:59:19.020531 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3046f29e-6945-42ff-a332-20319fac9713 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:59:19.107117 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-3046f29e-6945-42ff-a332-20319fac9713 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e28f30f382a4fb29b48ee708f8f44a9', 'request_id': 'req-3046f29e-6945-42ff-a332-20319fac9713'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:59:19.125308 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-3046f29e-6945-42ff-a332-20319fac9713 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e28f30f382a4fb29b48ee708f8f44a9', 'request_id': 'req-3046f29e-6945-42ff-a332-20319fac9713'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:59:19.141533 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-3046f29e-6945-42ff-a332-20319fac9713 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e28f30f382a4fb29b48ee708f8f44a9', 'request_id': 'req-3046f29e-6945-42ff-a332-20319fac9713'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:59:19.155986 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-3046f29e-6945-42ff-a332-20319fac9713 admin admin] The token cache is being invalidate because project 0e28f30f382a4fb29b48ee708f8f44a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90653) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:59:19.156383 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-3046f29e-6945-42ff-a332-20319fac9713 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:59:19.157345 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-3046f29e-6945-42ff-a332-20319fac9713 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:59:19.159138 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1382/2775] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:59:18 2022] DELETE /identity/v3/projects/0e28f30f382a4fb29b48ee708f8f44a9 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:59:19.164471 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0703baf1-5296-4b29-99ee-bd34af5115d6 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:19.165334 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0703baf1-5296-4b29-99ee-bd34af5115d6 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:19.165775 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0703baf1-5296-4b29-99ee-bd34af5115d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:19.211909 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-0703baf1-5296-4b29-99ee-bd34af5115d6 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:59:19.213754 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-0703baf1-5296-4b29-99ee-bd34af5115d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:59:19.284825 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1394/2776] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:59:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:59:19.293601 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad2de39-0f2e-4864-b9f2-043f3dd0c786 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:19.293969 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad2de39-0f2e-4864-b9f2-043f3dd0c786 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:19.294252 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad2de39-0f2e-4864-b9f2-043f3dd0c786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:19.302061 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:59:19.324087 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-fad2de39-0f2e-4864-b9f2-043f3dd0c786 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:59:19.326715 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-fad2de39-0f2e-4864-b9f2-043f3dd0c786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:59:19.338432 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1383/2777] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:59:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:59:19.343028 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0703baf1-5296-4b29-99ee-bd34af5115d6 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:59:19.356211 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0703baf1-5296-4b29-99ee-bd34af5115d6 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:59:19.365030 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f49ff13-80f8-4a08-8ed0-6fcc552c3c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:59:19.366915 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f49ff13-80f8-4a08-8ed0-6fcc552c3c96 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:19.368451 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f49ff13-80f8-4a08-8ed0-6fcc552c3c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:19.368451 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f49ff13-80f8-4a08-8ed0-6fcc552c3c96 admin admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:19.368947 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f49ff13-80f8-4a08-8ed0-6fcc552c3c96 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:59:19.369258 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f49ff13-80f8-4a08-8ed0-6fcc552c3c96 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:59:19.370875 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f49ff13-80f8-4a08-8ed0-6fcc552c3c96 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:59:19.386335 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1395/2778] 10.222.0.15 () {58 vars in 1153 bytes} [Sun Sep 11 10:59:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:59:19.390322 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad2de39-0f2e-4864-b9f2-043f3dd0c786 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:59:19.402612 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad2de39-0f2e-4864-b9f2-043f3dd0c786 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:59:19.410636 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5303400c-0afc-4dd1-8cff-f016354b492b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:59:19.411264 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5303400c-0afc-4dd1-8cff-f016354b492b admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:19.411579 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5303400c-0afc-4dd1-8cff-f016354b492b admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:19.411852 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5303400c-0afc-4dd1-8cff-f016354b492b admin admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:19.413242 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5303400c-0afc-4dd1-8cff-f016354b492b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:59:19.415206 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5303400c-0afc-4dd1-8cff-f016354b492b admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:59:19.455316 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1384/2779] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:59:19 2022] POST /identity/v3/projects => generated 375 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:59:19.460170 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f49ff13-80f8-4a08-8ed0-6fcc552c3c96 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:59:19.471722 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f49ff13-80f8-4a08-8ed0-6fcc552c3c96 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:59:19.543185 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:59:19.560874 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32f3660-0dfa-45ba-8928-7bb08a2691bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:59:19.561840 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32f3660-0dfa-45ba-8928-7bb08a2691bb admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:19.562663 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32f3660-0dfa-45ba-8928-7bb08a2691bb admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:19.563021 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32f3660-0dfa-45ba-8928-7bb08a2691bb admin admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:19.563726 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32f3660-0dfa-45ba-8928-7bb08a2691bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:59:19.566061 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32f3660-0dfa-45ba-8928-7bb08a2691bb admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:59:19.595595 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1396/2780] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:59:19 2022] POST /identity/v3/users => generated 288 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:59:19.601413 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5303400c-0afc-4dd1-8cff-f016354b492b admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:59:19.613992 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5303400c-0afc-4dd1-8cff-f016354b492b admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:59:19.620367 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b69393-8944-4815-b251-75db29900973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:59:19.620997 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b69393-8944-4815-b251-75db29900973 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:19.622044 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b69393-8944-4815-b251-75db29900973 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:19.622437 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b69393-8944-4815-b251-75db29900973 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:19.623358 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b69393-8944-4815-b251-75db29900973 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:59:19.625238 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b69393-8944-4815-b251-75db29900973 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:59:19.641023 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1385/2781] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:59:19 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:59:19.642669 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32f3660-0dfa-45ba-8928-7bb08a2691bb admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:59:19.654657 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32f3660-0dfa-45ba-8928-7bb08a2691bb admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:59:19.660902 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51252ee-ec3e-4a36-bd4d-3e548b7b2e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:59:19.662974 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51252ee-ec3e-4a36-bd4d-3e548b7b2e4c admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:19.664367 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51252ee-ec3e-4a36-bd4d-3e548b7b2e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:19.664879 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51252ee-ec3e-4a36-bd4d-3e548b7b2e4c admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:19.665742 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51252ee-ec3e-4a36-bd4d-3e548b7b2e4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:59:19.667824 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51252ee-ec3e-4a36-bd4d-3e548b7b2e4c admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:59:19.681549 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1397/2782] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:59:19 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:59:19.685558 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b69393-8944-4815-b251-75db29900973 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:59:19.696213 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b69393-8944-4815-b251-75db29900973 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:59:19.701112 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5ed687-094c-4a4d-87d4-920482cbf298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:59:19.701849 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ed687-094c-4a4d-87d4-920482cbf298 admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:19.702618 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ed687-094c-4a4d-87d4-920482cbf298 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:19.703203 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ed687-094c-4a4d-87d4-920482cbf298 admin admin] PATH_INFO: `/v3/projects/736e3e0a459c4f0a8062f05f6e4cff23/users/02c0dddd621b4977bcaa366e8ad0ac69/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:19.722586 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5ed687-094c-4a4d-87d4-920482cbf298 admin admin] RBAC: Authorizing `identity:create_grant(project_id=736e3e0a459c4f0a8062f05f6e4cff23, user_id=02c0dddd621b4977bcaa366e8ad0ac69, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:59:19.726706 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5ed687-094c-4a4d-87d4-920482cbf298 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:59:19.766244 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1386/2783] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:59:19 2022] PUT /identity/v3/projects/736e3e0a459c4f0a8062f05f6e4cff23/users/02c0dddd621b4977bcaa366e8ad0ac69/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:59:19.768137 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9948c4f1-f0af-44d2-bbdf-f22a0bdb2f40 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:19.768426 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9948c4f1-f0af-44d2-bbdf-f22a0bdb2f40 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:19.768701 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9948c4f1-f0af-44d2-bbdf-f22a0bdb2f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:19.800383 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-9948c4f1-f0af-44d2-bbdf-f22a0bdb2f40 None None] MFA Rules not processed for user `02c0dddd621b4977bcaa366e8ad0ac69`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:59:19.801742 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-9948c4f1-f0af-44d2-bbdf-f22a0bdb2f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:59:19.894438 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1398/2784] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:59:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 10:59:19.906243 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5ed687-094c-4a4d-87d4-920482cbf298 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:59:19.908115 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-6a5ed687-094c-4a4d-87d4-920482cbf298 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:59:19.974201 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5ed687-094c-4a4d-87d4-920482cbf298 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:59:19.981457 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25915408-5a69-46ac-bcc5-d04edbb70a25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:59:19.982306 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25915408-5a69-46ac-bcc5-d04edbb70a25 service cinder] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:19.982475 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25915408-5a69-46ac-bcc5-d04edbb70a25 service cinder] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:19.982754 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25915408-5a69-46ac-bcc5-d04edbb70a25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:19.997999 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25915408-5a69-46ac-bcc5-d04edbb70a25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:59:19.999478 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25915408-5a69-46ac-bcc5-d04edbb70a25 service cinder] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:59:20.024969 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1387/2785] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:59:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 10:59:20.329448 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6c65f5-2ad7-400c-bf2e-185db63ee849 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:20.330026 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6c65f5-2ad7-400c-bf2e-185db63ee849 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:20.330157 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6c65f5-2ad7-400c-bf2e-185db63ee849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:20.371458 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-7f6c65f5-2ad7-400c-bf2e-185db63ee849 None None] MFA Rules not processed for user `f071b2156180461aa0ee9c4c80acdc68`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:59:20.372994 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-7f6c65f5-2ad7-400c-bf2e-185db63ee849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:59:20.472023 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1399/2786] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:59:20 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:59:20.488239 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25915408-5a69-46ac-bcc5-d04edbb70a25 service cinder] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:59:20.489691 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-25915408-5a69-46ac-bcc5-d04edbb70a25 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:59:20.603220 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25915408-5a69-46ac-bcc5-d04edbb70a25 service cinder] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:59:20.608818 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4759b345-f0b0-4e72-ad1c-726acf40fb53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bd5356829b6454993bab9bad2f79fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:59:20.609537 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4759b345-f0b0-4e72-ad1c-726acf40fb53 service nova] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:20.609879 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4759b345-f0b0-4e72-ad1c-726acf40fb53 service nova] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:20.610223 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4759b345-f0b0-4e72-ad1c-726acf40fb53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:20.620554 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4759b345-f0b0-4e72-ad1c-726acf40fb53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:59:20.622231 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4759b345-f0b0-4e72-ad1c-726acf40fb53 service nova] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:59:20.631949 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1388/2787] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:59:20 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 10:59:21.955044 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0227c701-a8f0-432d-b09a-a4a200ca6789 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:21.955583 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0227c701-a8f0-432d-b09a-a4a200ca6789 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:21.955698 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0227c701-a8f0-432d-b09a-a4a200ca6789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:21.976351 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-0227c701-a8f0-432d-b09a-a4a200ca6789 None None] MFA Rules not processed for user `f071b2156180461aa0ee9c4c80acdc68`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:59:21.977679 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-0227c701-a8f0-432d-b09a-a4a200ca6789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:59:21.984793 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1400/2788] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:59:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:59:22.665223 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4759b345-f0b0-4e72-ad1c-726acf40fb53 service nova] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:59:22.667212 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-4759b345-f0b0-4e72-ad1c-726acf40fb53 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:59:22.701722 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:59:22.748738 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4759b345-f0b0-4e72-ad1c-726acf40fb53 service nova] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:59:22.753764 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba6c2a5-845a-4191-af45-5de49c5a8ac0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08477b4c88f04c9e97391aba401879e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:59:22.754436 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba6c2a5-845a-4191-af45-5de49c5a8ac0 service placement] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:22.754713 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba6c2a5-845a-4191-af45-5de49c5a8ac0 service placement] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:22.754981 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba6c2a5-845a-4191-af45-5de49c5a8ac0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:22.758059 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-3ba6c2a5-845a-4191-af45-5de49c5a8ac0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:59:22.772907 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba6c2a5-845a-4191-af45-5de49c5a8ac0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:59:22.774491 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba6c2a5-845a-4191-af45-5de49c5a8ac0 service placement] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:59:22.784731 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1389/2789] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:59:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:59:31.971073 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=75,keystone:INSERT=3 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:59:32.780425 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=20 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:59:42.573757 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0227c701-a8f0-432d-b09a-a4a200ca6789 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:59:42.575324 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-0227c701-a8f0-432d-b09a-a4a200ca6789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:59:42.595587 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0227c701-a8f0-432d-b09a-a4a200ca6789 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:59:42.601318 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78423f69-51cd-42d7-b724-228b05be37fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bd5356829b6454993bab9bad2f79fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:59:42.601930 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78423f69-51cd-42d7-b724-228b05be37fd service nova] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:42.603255 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78423f69-51cd-42d7-b724-228b05be37fd service nova] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:42.603255 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78423f69-51cd-42d7-b724-228b05be37fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:42.605280 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-78423f69-51cd-42d7-b724-228b05be37fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:59:42.618618 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78423f69-51cd-42d7-b724-228b05be37fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:59:42.620434 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78423f69-51cd-42d7-b724-228b05be37fd service nova] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:59:42.631543 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1401/2790] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:59:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:59:45.195657 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba6c2a5-845a-4191-af45-5de49c5a8ac0 service placement] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:59:45.258175 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba6c2a5-845a-4191-af45-5de49c5a8ac0 service placement] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:59:45.263263 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2e0450-9147-4802-8070-10088ae12419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:59:45.263937 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2e0450-9147-4802-8070-10088ae12419 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:45.264220 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2e0450-9147-4802-8070-10088ae12419 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:45.264492 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2e0450-9147-4802-8070-10088ae12419 admin admin] PATH_INFO: `/v3/users/02c0dddd621b4977bcaa366e8ad0ac69` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:45.265592 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2e0450-9147-4802-8070-10088ae12419 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02c0dddd621b4977bcaa366e8ad0ac69)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:59:45.267539 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2e0450-9147-4802-8070-10088ae12419 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:59:45.314215 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-bf2e0450-9147-4802-8070-10088ae12419 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02c0dddd621b4977bcaa366e8ad0ac69'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:59:45.322442 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-bf2e0450-9147-4802-8070-10088ae12419 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02c0dddd621b4977bcaa366e8ad0ac69'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:59:45.323020 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-bf2e0450-9147-4802-8070-10088ae12419 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02c0dddd621b4977bcaa366e8ad0ac69'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:59:45.323656 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-bf2e0450-9147-4802-8070-10088ae12419 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02c0dddd621b4977bcaa366e8ad0ac69'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:59:45.330731 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-bf2e0450-9147-4802-8070-10088ae12419 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02c0dddd621b4977bcaa366e8ad0ac69'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:59:45.350137 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1390/2791] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 10:59:45 2022] DELETE /identity/v3/users/02c0dddd621b4977bcaa366e8ad0ac69 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:59:45.353657 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9df93ee-84ae-4050-b581-60bdde8a9ee1 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:45.354138 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9df93ee-84ae-4050-b581-60bdde8a9ee1 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:45.354248 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9df93ee-84ae-4050-b581-60bdde8a9ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:45.373188 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-f9df93ee-84ae-4050-b581-60bdde8a9ee1 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:59:45.374385 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-f9df93ee-84ae-4050-b581-60bdde8a9ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:59:45.431665 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1402/2792] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:59:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:59:45.441202 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2e0450-9147-4802-8070-10088ae12419 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:59:45.442598 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-bf2e0450-9147-4802-8070-10088ae12419 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:59:45.500105 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2e0450-9147-4802-8070-10088ae12419 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:59:45.503775 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d3fea3-9b7e-41f4-bed1-f6213f0e3098 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:59:45.504377 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d3fea3-9b7e-41f4-bed1-f6213f0e3098 service neutron] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:45.504687 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d3fea3-9b7e-41f4-bed1-f6213f0e3098 service neutron] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:45.504963 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d3fea3-9b7e-41f4-bed1-f6213f0e3098 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:45.515023 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d3fea3-9b7e-41f4-bed1-f6213f0e3098 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:59:45.516610 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d3fea3-9b7e-41f4-bed1-f6213f0e3098 service neutron] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:59:45.526796 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1391/2793] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:59:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:59:45.789375 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9df93ee-84ae-4050-b581-60bdde8a9ee1 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:59:45.791154 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-f9df93ee-84ae-4050-b581-60bdde8a9ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:59:45.805389 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9df93ee-84ae-4050-b581-60bdde8a9ee1 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:59:45.810947 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca317642-153e-4ac4-9dce-f377274e0bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:59:45.811618 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca317642-153e-4ac4-9dce-f377274e0bb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:45.811916 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca317642-153e-4ac4-9dce-f377274e0bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:45.812161 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca317642-153e-4ac4-9dce-f377274e0bb4 admin admin] PATH_INFO: `/v3/projects/736e3e0a459c4f0a8062f05f6e4cff23` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:45.813276 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca317642-153e-4ac4-9dce-f377274e0bb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=736e3e0a459c4f0a8062f05f6e4cff23)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:59:45.815189 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca317642-153e-4ac4-9dce-f377274e0bb4 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:59:45.863948 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-ca317642-153e-4ac4-9dce-f377274e0bb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '736e3e0a459c4f0a8062f05f6e4cff23', 'request_id': 'req-ca317642-153e-4ac4-9dce-f377274e0bb4'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:59:45.871511 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-ca317642-153e-4ac4-9dce-f377274e0bb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '736e3e0a459c4f0a8062f05f6e4cff23', 'request_id': 'req-ca317642-153e-4ac4-9dce-f377274e0bb4'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:59:45.877406 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-ca317642-153e-4ac4-9dce-f377274e0bb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '736e3e0a459c4f0a8062f05f6e4cff23', 'request_id': 'req-ca317642-153e-4ac4-9dce-f377274e0bb4'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:59:45.889927 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-ca317642-153e-4ac4-9dce-f377274e0bb4 admin admin] The token cache is being invalidate because project 736e3e0a459c4f0a8062f05f6e4cff23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 10:59:45.890211 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-ca317642-153e-4ac4-9dce-f377274e0bb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:59:45.890863 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-ca317642-153e-4ac4-9dce-f377274e0bb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 10:59:45.892374 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1403/2794] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 10:59:45 2022] DELETE /identity/v3/projects/736e3e0a459c4f0a8062f05f6e4cff23 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:59:45.897292 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02392bf-53cf-4951-b476-bb2b96374292 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:45.897590 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02392bf-53cf-4951-b476-bb2b96374292 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:45.897867 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02392bf-53cf-4951-b476-bb2b96374292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:45.919092 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-a02392bf-53cf-4951-b476-bb2b96374292 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:59:45.920346 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-a02392bf-53cf-4951-b476-bb2b96374292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:59:45.940053 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:59:45.988579 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1392/2795] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:59:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:59:45.997477 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a6319-6a6c-4709-8bbf-64fafae590aa None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:45.997763 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a6319-6a6c-4709-8bbf-64fafae590aa None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:45.998039 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a6319-6a6c-4709-8bbf-64fafae590aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:46.018104 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-341a6319-6a6c-4709-8bbf-64fafae590aa None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:59:46.019638 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-341a6319-6a6c-4709-8bbf-64fafae590aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:59:46.028207 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1404/2796] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:59:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:59:46.032711 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02392bf-53cf-4951-b476-bb2b96374292 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:59:46.043322 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02392bf-53cf-4951-b476-bb2b96374292 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:59:46.047907 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a538d950-62f9-47d7-b002-ad70e368a4cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:59:46.048508 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a538d950-62f9-47d7-b002-ad70e368a4cc admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:46.048794 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a538d950-62f9-47d7-b002-ad70e368a4cc admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:46.049062 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a538d950-62f9-47d7-b002-ad70e368a4cc admin admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:46.049703 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a538d950-62f9-47d7-b002-ad70e368a4cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 10:59:46.050009 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a538d950-62f9-47d7-b002-ad70e368a4cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:59:46.051665 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a538d950-62f9-47d7-b002-ad70e368a4cc admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:59:46.063874 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1393/2797] 10.222.0.15 () {58 vars in 1153 bytes} [Sun Sep 11 10:59:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 10:59:46.068330 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341a6319-6a6c-4709-8bbf-64fafae590aa None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:59:46.078127 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341a6319-6a6c-4709-8bbf-64fafae590aa None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:59:46.082856 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7599e8a8-fa2c-4321-82ae-181a45851133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:59:46.083587 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7599e8a8-fa2c-4321-82ae-181a45851133 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:46.083931 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7599e8a8-fa2c-4321-82ae-181a45851133 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:46.084270 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7599e8a8-fa2c-4321-82ae-181a45851133 admin admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:46.084972 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7599e8a8-fa2c-4321-82ae-181a45851133 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:59:46.087121 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7599e8a8-fa2c-4321-82ae-181a45851133 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:59:46.121510 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1405/2798] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 10:59:46 2022] POST /identity/v3/projects => generated 363 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:59:46.122522 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a538d950-62f9-47d7-b002-ad70e368a4cc admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:59:46.132617 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a538d950-62f9-47d7-b002-ad70e368a4cc admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:59:46.191472 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c104a1-93fa-47c8-b385-92fc9dde36c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:59:46.192087 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c104a1-93fa-47c8-b385-92fc9dde36c8 admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:46.192366 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c104a1-93fa-47c8-b385-92fc9dde36c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:46.192650 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c104a1-93fa-47c8-b385-92fc9dde36c8 admin admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:46.193281 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c104a1-93fa-47c8-b385-92fc9dde36c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:59:46.195212 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c104a1-93fa-47c8-b385-92fc9dde36c8 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:59:46.217864 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1394/2799] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 10:59:46 2022] POST /identity/v3/users => generated 282 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 10:59:46.221903 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7599e8a8-fa2c-4321-82ae-181a45851133 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:59:46.234733 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7599e8a8-fa2c-4321-82ae-181a45851133 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:59:46.239822 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0618bc04-cbfe-428e-ac00-bb65dae6744b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:59:46.240502 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0618bc04-cbfe-428e-ac00-bb65dae6744b admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:46.240904 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0618bc04-cbfe-428e-ac00-bb65dae6744b admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:46.241241 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0618bc04-cbfe-428e-ac00-bb65dae6744b admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:46.241953 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0618bc04-cbfe-428e-ac00-bb65dae6744b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:59:46.244684 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0618bc04-cbfe-428e-ac00-bb65dae6744b admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:59:46.254958 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1406/2800] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:59:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:59:46.259145 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c104a1-93fa-47c8-b385-92fc9dde36c8 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:59:46.269255 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c104a1-93fa-47c8-b385-92fc9dde36c8 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:59:46.273885 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd37cfcd-062e-4518-92a6-edefd15027c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:59:46.275015 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd37cfcd-062e-4518-92a6-edefd15027c5 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:46.275623 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd37cfcd-062e-4518-92a6-edefd15027c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:46.275916 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd37cfcd-062e-4518-92a6-edefd15027c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:46.276553 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd37cfcd-062e-4518-92a6-edefd15027c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:59:46.278158 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd37cfcd-062e-4518-92a6-edefd15027c5 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:59:46.289430 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1395/2801] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 10:59:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 10:59:46.294218 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0618bc04-cbfe-428e-ac00-bb65dae6744b admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:59:46.305166 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0618bc04-cbfe-428e-ac00-bb65dae6744b admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:59:46.310208 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e220a4-2a83-4e35-a17a-88b25b3deaa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:59:46.310877 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e220a4-2a83-4e35-a17a-88b25b3deaa1 admin admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:46.311165 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e220a4-2a83-4e35-a17a-88b25b3deaa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:46.311468 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e220a4-2a83-4e35-a17a-88b25b3deaa1 admin admin] PATH_INFO: `/v3/projects/dd59321c528049bc929eb6a8f00bd46e/users/6952467a87c341e0b779059615b785bd/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:46.327185 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e220a4-2a83-4e35-a17a-88b25b3deaa1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd59321c528049bc929eb6a8f00bd46e, user_id=6952467a87c341e0b779059615b785bd, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:59:46.329913 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e220a4-2a83-4e35-a17a-88b25b3deaa1 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:59:46.343881 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1407/2802] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 10:59:46 2022] PUT /identity/v3/projects/dd59321c528049bc929eb6a8f00bd46e/users/6952467a87c341e0b779059615b785bd/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 10:59:46.413992 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889c4ab0-fbc7-46a2-9c6a-0bede8cc37cf None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:46.414206 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889c4ab0-fbc7-46a2-9c6a-0bede8cc37cf None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:46.414458 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889c4ab0-fbc7-46a2-9c6a-0bede8cc37cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:46.443458 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-889c4ab0-fbc7-46a2-9c6a-0bede8cc37cf None None] MFA Rules not processed for user `6952467a87c341e0b779059615b785bd`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 10:59:46.444704 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-889c4ab0-fbc7-46a2-9c6a-0bede8cc37cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:59:46.514363 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1396/2803] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 10:59:46 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 10:59:46.526542 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e220a4-2a83-4e35-a17a-88b25b3deaa1 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:59:46.528257 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-c9e220a4-2a83-4e35-a17a-88b25b3deaa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:59:46.537947 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:59:46.589860 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e220a4-2a83-4e35-a17a-88b25b3deaa1 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:59:46.593543 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735be0df-824c-4d39-a8bb-43d2c8db8425 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:59:46.594150 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735be0df-824c-4d39-a8bb-43d2c8db8425 service cinder] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:46.594431 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735be0df-824c-4d39-a8bb-43d2c8db8425 service cinder] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:46.594704 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735be0df-824c-4d39-a8bb-43d2c8db8425 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:46.603091 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735be0df-824c-4d39-a8bb-43d2c8db8425 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:59:46.604563 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735be0df-824c-4d39-a8bb-43d2c8db8425 service cinder] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:59:46.613618 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1408/2804] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:59:46 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:59:47.311906 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889c4ab0-fbc7-46a2-9c6a-0bede8cc37cf None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:59:47.313408 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-889c4ab0-fbc7-46a2-9c6a-0bede8cc37cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:59:47.333827 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 10:59:47.389449 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889c4ab0-fbc7-46a2-9c6a-0bede8cc37cf None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:59:47.394903 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf5b694-7873-4fd2-b5e5-0979f04afc90 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bd5356829b6454993bab9bad2f79fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:59:47.395538 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf5b694-7873-4fd2-b5e5-0979f04afc90 service nova] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:47.395822 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf5b694-7873-4fd2-b5e5-0979f04afc90 service nova] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:47.396094 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf5b694-7873-4fd2-b5e5-0979f04afc90 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:47.398335 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-baf5b694-7873-4fd2-b5e5-0979f04afc90 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:59:47.411447 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf5b694-7873-4fd2-b5e5-0979f04afc90 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:59:47.413065 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf5b694-7873-4fd2-b5e5-0979f04afc90 service nova] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:59:47.424467 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1397/2805] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:59:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:59:51.069496 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735be0df-824c-4d39-a8bb-43d2c8db8425 service cinder] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 10:59:51.070895 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-735be0df-824c-4d39-a8bb-43d2c8db8425 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:59:51.144341 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735be0df-824c-4d39-a8bb-43d2c8db8425 service cinder] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 10:59:51.150425 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43911572-ea4e-4a37-8d25-29255a99688e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08477b4c88f04c9e97391aba401879e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 10:59:51.151374 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43911572-ea4e-4a37-8d25-29255a99688e service placement] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 10:59:51.151716 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43911572-ea4e-4a37-8d25-29255a99688e service placement] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 10:59:51.151990 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43911572-ea4e-4a37-8d25-29255a99688e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 10:59:51.153756 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-43911572-ea4e-4a37-8d25-29255a99688e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 10:59:51.165391 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43911572-ea4e-4a37-8d25-29255a99688e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 10:59:51.167708 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43911572-ea4e-4a37-8d25-29255a99688e service placement] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 10:59:51.178280 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1409/2806] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 10:59:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 10:59:57.420602 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=26 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 11:00:01.173604 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=55 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 11:00:34.526088 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf5b694-7873-4fd2-b5e5-0979f04afc90 service nova] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:00:34.600325 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf5b694-7873-4fd2-b5e5-0979f04afc90 service nova] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:00:34.604879 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0548c889-3561-4f3d-8c74-616606d7bfaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:00:34.605512 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0548c889-3561-4f3d-8c74-616606d7bfaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:00:34.605790 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0548c889-3561-4f3d-8c74-616606d7bfaa admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:00:34.606063 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0548c889-3561-4f3d-8c74-616606d7bfaa admin admin] PATH_INFO: `/v3/users/6952467a87c341e0b779059615b785bd` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:00:34.607179 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0548c889-3561-4f3d-8c74-616606d7bfaa admin admin] RBAC: Authorizing `identity:delete_user(user_id=6952467a87c341e0b779059615b785bd)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:00:34.609128 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0548c889-3561-4f3d-8c74-616606d7bfaa admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:00:34.654721 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-0548c889-3561-4f3d-8c74-616606d7bfaa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6952467a87c341e0b779059615b785bd'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:00:34.661762 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-0548c889-3561-4f3d-8c74-616606d7bfaa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6952467a87c341e0b779059615b785bd'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:00:34.662379 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-0548c889-3561-4f3d-8c74-616606d7bfaa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6952467a87c341e0b779059615b785bd'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:00:34.662977 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-0548c889-3561-4f3d-8c74-616606d7bfaa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6952467a87c341e0b779059615b785bd'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:00:34.669936 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-0548c889-3561-4f3d-8c74-616606d7bfaa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6952467a87c341e0b779059615b785bd'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:00:34.687161 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1398/2807] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 11:00:34 2022] DELETE /identity/v3/users/6952467a87c341e0b779059615b785bd => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 11:00:34.692506 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce78fa3c-1ed2-4bf8-bd6f-012645d65805 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:00:34.693015 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce78fa3c-1ed2-4bf8-bd6f-012645d65805 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:00:34.693137 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce78fa3c-1ed2-4bf8-bd6f-012645d65805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:00:34.711685 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-ce78fa3c-1ed2-4bf8-bd6f-012645d65805 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 11:00:34.712867 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-ce78fa3c-1ed2-4bf8-bd6f-012645d65805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:00:34.773988 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1410/2808] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 11:00:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 11:00:34.784179 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0548c889-3561-4f3d-8c74-616606d7bfaa admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:00:34.785586 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-0548c889-3561-4f3d-8c74-616606d7bfaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:00:34.849063 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0548c889-3561-4f3d-8c74-616606d7bfaa admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:00:34.852700 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9bd69b-83e7-4a82-bf8d-332e7cb72494 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:00:34.853303 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9bd69b-83e7-4a82-bf8d-332e7cb72494 service neutron] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:00:34.853580 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9bd69b-83e7-4a82-bf8d-332e7cb72494 service neutron] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:00:34.853844 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9bd69b-83e7-4a82-bf8d-332e7cb72494 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:00:34.864068 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9bd69b-83e7-4a82-bf8d-332e7cb72494 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:00:34.865474 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9bd69b-83e7-4a82-bf8d-332e7cb72494 service neutron] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:00:34.874784 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1399/2809] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 11:00:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 11:00:35.141793 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce78fa3c-1ed2-4bf8-bd6f-012645d65805 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:00:35.143325 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-ce78fa3c-1ed2-4bf8-bd6f-012645d65805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:00:35.155234 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce78fa3c-1ed2-4bf8-bd6f-012645d65805 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:00:35.160766 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5123fcb3-eafb-4c09-a949-ced7cb5a3aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:00:35.161428 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5123fcb3-eafb-4c09-a949-ced7cb5a3aa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:00:35.161711 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5123fcb3-eafb-4c09-a949-ced7cb5a3aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:00:35.161988 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5123fcb3-eafb-4c09-a949-ced7cb5a3aa2 admin admin] PATH_INFO: `/v3/projects/dd59321c528049bc929eb6a8f00bd46e` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:00:35.163124 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5123fcb3-eafb-4c09-a949-ced7cb5a3aa2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd59321c528049bc929eb6a8f00bd46e)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:00:35.165099 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5123fcb3-eafb-4c09-a949-ced7cb5a3aa2 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:00:35.212385 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-5123fcb3-eafb-4c09-a949-ced7cb5a3aa2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd59321c528049bc929eb6a8f00bd46e', 'request_id': 'req-5123fcb3-eafb-4c09-a949-ced7cb5a3aa2'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:00:35.218572 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-5123fcb3-eafb-4c09-a949-ced7cb5a3aa2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd59321c528049bc929eb6a8f00bd46e', 'request_id': 'req-5123fcb3-eafb-4c09-a949-ced7cb5a3aa2'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:00:35.225589 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-5123fcb3-eafb-4c09-a949-ced7cb5a3aa2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd59321c528049bc929eb6a8f00bd46e', 'request_id': 'req-5123fcb3-eafb-4c09-a949-ced7cb5a3aa2'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:00:35.238302 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-5123fcb3-eafb-4c09-a949-ced7cb5a3aa2 admin admin] The token cache is being invalidate because project dd59321c528049bc929eb6a8f00bd46e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 11:00:35.238628 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-5123fcb3-eafb-4c09-a949-ced7cb5a3aa2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:00:35.239263 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-5123fcb3-eafb-4c09-a949-ced7cb5a3aa2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:00:35.240782 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1411/2810] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 11:00:35 2022] DELETE /identity/v3/projects/dd59321c528049bc929eb6a8f00bd46e => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 11:00:35.825548 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9bd69b-83e7-4a82-bf8d-332e7cb72494 service neutron] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:00:35.827061 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-5a9bd69b-83e7-4a82-bf8d-332e7cb72494 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:00:35.841034 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9bd69b-83e7-4a82-bf8d-332e7cb72494 service neutron] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:00:35.859626 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 11:00:35.892148 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08a777e-d578-4960-bf20-dac909fd0c35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:00:35.892770 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08a777e-d578-4960-bf20-dac909fd0c35 service neutron] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:00:35.893049 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08a777e-d578-4960-bf20-dac909fd0c35 service neutron] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:00:35.893319 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08a777e-d578-4960-bf20-dac909fd0c35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:00:35.895039 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-f08a777e-d578-4960-bf20-dac909fd0c35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:00:35.957065 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08a777e-d578-4960-bf20-dac909fd0c35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:00:35.958490 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08a777e-d578-4960-bf20-dac909fd0c35 service neutron] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:00:35.968067 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1400/2811] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 11:00:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 11:00:42.067611 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5123fcb3-eafb-4c09-a949-ced7cb5a3aa2 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:00:42.072126 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-5123fcb3-eafb-4c09-a949-ced7cb5a3aa2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:00:42.134826 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08a777e-d578-4960-bf20-dac909fd0c35 service neutron] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:00:42.137413 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-f08a777e-d578-4960-bf20-dac909fd0c35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:00:42.151728 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08a777e-d578-4960-bf20-dac909fd0c35 service neutron] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:00:42.157762 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3f1ef7-17cf-4590-b9cd-bb1fa636c1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:00:42.158587 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f1ef7-17cf-4590-b9cd-bb1fa636c1a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:00:42.158986 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f1ef7-17cf-4590-b9cd-bb1fa636c1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:00:42.159386 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f1ef7-17cf-4590-b9cd-bb1fa636c1a9 admin admin] PATH_INFO: `/v3/users/e1b0815853b947bc8f50d91c1ce0ab5a` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:00:42.167319 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5123fcb3-eafb-4c09-a949-ced7cb5a3aa2 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:00:42.174357 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80c4aea-e73e-4fd1-9917-7592c7237779 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08477b4c88f04c9e97391aba401879e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:00:42.175050 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80c4aea-e73e-4fd1-9917-7592c7237779 service placement] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:00:42.175355 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80c4aea-e73e-4fd1-9917-7592c7237779 service placement] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:00:42.175664 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80c4aea-e73e-4fd1-9917-7592c7237779 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:00:42.177291 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3f1ef7-17cf-4590-b9cd-bb1fa636c1a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1b0815853b947bc8f50d91c1ce0ab5a)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:00:42.177423 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-b80c4aea-e73e-4fd1-9917-7592c7237779 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:00:42.179339 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3f1ef7-17cf-4590-b9cd-bb1fa636c1a9 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:00:42.194462 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80c4aea-e73e-4fd1-9917-7592c7237779 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:00:42.196102 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80c4aea-e73e-4fd1-9917-7592c7237779 service placement] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:00:42.206751 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1412/2812] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 11:00:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 11:00:42.232115 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-5c3f1ef7-17cf-4590-b9cd-bb1fa636c1a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1b0815853b947bc8f50d91c1ce0ab5a'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:00:42.240526 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-5c3f1ef7-17cf-4590-b9cd-bb1fa636c1a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1b0815853b947bc8f50d91c1ce0ab5a'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:00:42.241203 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-5c3f1ef7-17cf-4590-b9cd-bb1fa636c1a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1b0815853b947bc8f50d91c1ce0ab5a'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:00:42.241880 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-5c3f1ef7-17cf-4590-b9cd-bb1fa636c1a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1b0815853b947bc8f50d91c1ce0ab5a'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:00:42.249045 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-5c3f1ef7-17cf-4590-b9cd-bb1fa636c1a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1b0815853b947bc8f50d91c1ce0ab5a'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:00:42.266780 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1401/2813] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 11:00:42 2022] DELETE /identity/v3/users/e1b0815853b947bc8f50d91c1ce0ab5a => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 11:00:42.437311 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80c4aea-e73e-4fd1-9917-7592c7237779 service placement] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:00:42.448721 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80c4aea-e73e-4fd1-9917-7592c7237779 service placement] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:00:42.461690 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 11:00:42.513103 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15521c7-bf82-4e4c-bddb-2be1cb8e1751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:00:42.513738 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15521c7-bf82-4e4c-bddb-2be1cb8e1751 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:00:42.514020 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15521c7-bf82-4e4c-bddb-2be1cb8e1751 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:00:42.514313 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15521c7-bf82-4e4c-bddb-2be1cb8e1751 admin admin] PATH_INFO: `/v3/projects/23556239691f46aa8b39bb6219fc591f` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:00:42.525232 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15521c7-bf82-4e4c-bddb-2be1cb8e1751 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23556239691f46aa8b39bb6219fc591f)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:00:42.527144 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15521c7-bf82-4e4c-bddb-2be1cb8e1751 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:00:42.571902 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-a15521c7-bf82-4e4c-bddb-2be1cb8e1751 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23556239691f46aa8b39bb6219fc591f', 'request_id': 'req-a15521c7-bf82-4e4c-bddb-2be1cb8e1751'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:00:42.578404 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-a15521c7-bf82-4e4c-bddb-2be1cb8e1751 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23556239691f46aa8b39bb6219fc591f', 'request_id': 'req-a15521c7-bf82-4e4c-bddb-2be1cb8e1751'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:00:42.584911 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-a15521c7-bf82-4e4c-bddb-2be1cb8e1751 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23556239691f46aa8b39bb6219fc591f', 'request_id': 'req-a15521c7-bf82-4e4c-bddb-2be1cb8e1751'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:00:42.596723 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-a15521c7-bf82-4e4c-bddb-2be1cb8e1751 admin admin] The token cache is being invalidate because project 23556239691f46aa8b39bb6219fc591f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 11:00:42.597010 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-a15521c7-bf82-4e4c-bddb-2be1cb8e1751 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:00:42.597622 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-a15521c7-bf82-4e4c-bddb-2be1cb8e1751 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:00:42.599083 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1413/2814] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 11:00:42 2022] DELETE /identity/v3/projects/23556239691f46aa8b39bb6219fc591f => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 11:00:42.603192 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3f1ef7-17cf-4590-b9cd-bb1fa636c1a9 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:00:42.604607 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-5c3f1ef7-17cf-4590-b9cd-bb1fa636c1a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:00:42.617878 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3f1ef7-17cf-4590-b9cd-bb1fa636c1a9 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:00:42.648123 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 11:00:42.676610 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bba438e-f5d2-4afc-8ff9-e56631d53fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:00:42.677241 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bba438e-f5d2-4afc-8ff9-e56631d53fe4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:00:42.677523 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bba438e-f5d2-4afc-8ff9-e56631d53fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:00:42.677798 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bba438e-f5d2-4afc-8ff9-e56631d53fe4 admin admin] PATH_INFO: `/v3/users/bbd3041dc72e4db29a7eccb276e00de4` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:00:42.691270 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bba438e-f5d2-4afc-8ff9-e56631d53fe4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbd3041dc72e4db29a7eccb276e00de4)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:00:42.693142 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bba438e-f5d2-4afc-8ff9-e56631d53fe4 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:00:42.738297 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-8bba438e-f5d2-4afc-8ff9-e56631d53fe4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbd3041dc72e4db29a7eccb276e00de4'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:00:42.745354 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-8bba438e-f5d2-4afc-8ff9-e56631d53fe4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbd3041dc72e4db29a7eccb276e00de4'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:00:42.745995 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-8bba438e-f5d2-4afc-8ff9-e56631d53fe4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbd3041dc72e4db29a7eccb276e00de4'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:00:42.746620 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-8bba438e-f5d2-4afc-8ff9-e56631d53fe4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbd3041dc72e4db29a7eccb276e00de4'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:00:42.754479 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-8bba438e-f5d2-4afc-8ff9-e56631d53fe4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbd3041dc72e4db29a7eccb276e00de4'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:00:42.773575 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1402/2815] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 11:00:42 2022] DELETE /identity/v3/users/bbd3041dc72e4db29a7eccb276e00de4 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 11:00:42.946736 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15521c7-bf82-4e4c-bddb-2be1cb8e1751 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:00:42.948328 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-a15521c7-bf82-4e4c-bddb-2be1cb8e1751 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:00:42.963132 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15521c7-bf82-4e4c-bddb-2be1cb8e1751 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:00:43.020392 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cfadbc-a80d-454a-b1c0-4426ec4bd497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:00:43.021023 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cfadbc-a80d-454a-b1c0-4426ec4bd497 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:00:43.021302 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cfadbc-a80d-454a-b1c0-4426ec4bd497 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:00:43.021574 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cfadbc-a80d-454a-b1c0-4426ec4bd497 admin admin] PATH_INFO: `/v3/projects/2ded14adc3704a139c3e310030b1f5bc` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:00:43.031762 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cfadbc-a80d-454a-b1c0-4426ec4bd497 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ded14adc3704a139c3e310030b1f5bc)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:00:43.034398 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cfadbc-a80d-454a-b1c0-4426ec4bd497 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:00:43.065232 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 11:00:43.081947 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-33cfadbc-a80d-454a-b1c0-4426ec4bd497 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ded14adc3704a139c3e310030b1f5bc', 'request_id': 'req-33cfadbc-a80d-454a-b1c0-4426ec4bd497'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:00:43.088643 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-33cfadbc-a80d-454a-b1c0-4426ec4bd497 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ded14adc3704a139c3e310030b1f5bc', 'request_id': 'req-33cfadbc-a80d-454a-b1c0-4426ec4bd497'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:00:43.094027 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-33cfadbc-a80d-454a-b1c0-4426ec4bd497 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ded14adc3704a139c3e310030b1f5bc', 'request_id': 'req-33cfadbc-a80d-454a-b1c0-4426ec4bd497'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:00:43.105080 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-33cfadbc-a80d-454a-b1c0-4426ec4bd497 admin admin] The token cache is being invalidate because project 2ded14adc3704a139c3e310030b1f5bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 11:00:43.105366 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-33cfadbc-a80d-454a-b1c0-4426ec4bd497 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:00:43.105977 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-33cfadbc-a80d-454a-b1c0-4426ec4bd497 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:00:43.107440 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1414/2816] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 11:00:42 2022] DELETE /identity/v3/projects/2ded14adc3704a139c3e310030b1f5bc => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 11:00:52.768414 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=43,keystone:DELETE=10,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 11:00:53.102203 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=14,keystone:DELETE=5,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 11:01:27.405317 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ebd4bc-0a30-4cbb-86f0-e6b5656d203f None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:01:27.405862 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ebd4bc-0a30-4cbb-86f0-e6b5656d203f None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:01:27.405862 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ebd4bc-0a30-4cbb-86f0-e6b5656d203f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:01:27.425309 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-36ebd4bc-0a30-4cbb-86f0-e6b5656d203f None None] MFA Rules not processed for user `f071b2156180461aa0ee9c4c80acdc68`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 11:01:27.427188 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-36ebd4bc-0a30-4cbb-86f0-e6b5656d203f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:01:27.480184 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1403/2817] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 11:01:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 11:01:37.473003 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=25 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 11:01:37.657437 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cfadbc-a80d-454a-b1c0-4426ec4bd497 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:01:37.658852 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-33cfadbc-a80d-454a-b1c0-4426ec4bd497 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:01:37.714575 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cfadbc-a80d-454a-b1c0-4426ec4bd497 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:01:37.718144 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550aae4b-2999-4dbc-bed2-616fd9bad936 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:01:37.718744 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550aae4b-2999-4dbc-bed2-616fd9bad936 service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:01:37.719020 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550aae4b-2999-4dbc-bed2-616fd9bad936 service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:01:37.719292 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550aae4b-2999-4dbc-bed2-616fd9bad936 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:01:37.721012 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-550aae4b-2999-4dbc-bed2-616fd9bad936 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:01:37.790394 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550aae4b-2999-4dbc-bed2-616fd9bad936 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:01:37.791867 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550aae4b-2999-4dbc-bed2-616fd9bad936 service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:01:37.803229 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1415/2818] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 11:01:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 11:01:47.797762 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=54 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 11:02:54.397714 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ebd4bc-0a30-4cbb-86f0-e6b5656d203f None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:02:54.399100 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-36ebd4bc-0a30-4cbb-86f0-e6b5656d203f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:02:54.458863 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ebd4bc-0a30-4cbb-86f0-e6b5656d203f None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:02:54.462694 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feff9e9-94f6-4f46-8e64-1d8edb4901a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:02:54.463291 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feff9e9-94f6-4f46-8e64-1d8edb4901a9 service cinder] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:02:54.463604 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feff9e9-94f6-4f46-8e64-1d8edb4901a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:02:54.463872 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feff9e9-94f6-4f46-8e64-1d8edb4901a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:02:54.465595 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-6feff9e9-94f6-4f46-8e64-1d8edb4901a9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:02:54.479597 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feff9e9-94f6-4f46-8e64-1d8edb4901a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:02:54.480998 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feff9e9-94f6-4f46-8e64-1d8edb4901a9 service cinder] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:02:54.492811 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1404/2819] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 11:02:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 11:02:57.344191 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550aae4b-2999-4dbc-bed2-616fd9bad936 service neutron] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:02:57.346316 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-550aae4b-2999-4dbc-bed2-616fd9bad936 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:02:57.366918 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550aae4b-2999-4dbc-bed2-616fd9bad936 service neutron] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:02:57.378794 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fa8daa-3507-4f64-aad8-e9037e82b2c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bd5356829b6454993bab9bad2f79fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:02:57.379726 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fa8daa-3507-4f64-aad8-e9037e82b2c6 service nova] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:02:57.380668 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fa8daa-3507-4f64-aad8-e9037e82b2c6 service nova] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:02:57.381321 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fa8daa-3507-4f64-aad8-e9037e82b2c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:02:57.383455 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-39fa8daa-3507-4f64-aad8-e9037e82b2c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:02:57.398309 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fa8daa-3507-4f64-aad8-e9037e82b2c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:02:57.399957 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fa8daa-3507-4f64-aad8-e9037e82b2c6 service nova] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:02:57.412025 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1416/2820] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 11:02:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 11:03:04.487266 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=26 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 11:03:05.887964 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d143e6b3-8ab8-4bfd-ab76-bbb97d6ab7b1 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:03:05.888659 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d143e6b3-8ab8-4bfd-ab76-bbb97d6ab7b1 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:03:05.888659 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d143e6b3-8ab8-4bfd-ab76-bbb97d6ab7b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:03:05.907727 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-d143e6b3-8ab8-4bfd-ab76-bbb97d6ab7b1 None None] MFA Rules not processed for user `f071b2156180461aa0ee9c4c80acdc68`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 11:03:05.908928 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-d143e6b3-8ab8-4bfd-ab76-bbb97d6ab7b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:03:05.916548 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1405/2821] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 11:03:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 11:03:06.220922 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fa8daa-3507-4f64-aad8-e9037e82b2c6 service nova] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:03:06.230556 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fa8daa-3507-4f64-aad8-e9037e82b2c6 service nova] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:03:06.234121 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085c85b0-79cc-4f00-ae0d-65c41cd89c2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:03:06.234713 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c85b0-79cc-4f00-ae0d-65c41cd89c2a service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:03:06.234996 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c85b0-79cc-4f00-ae0d-65c41cd89c2a service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:03:06.235270 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c85b0-79cc-4f00-ae0d-65c41cd89c2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:03:06.237254 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-085c85b0-79cc-4f00-ae0d-65c41cd89c2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:03:06.246802 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085c85b0-79cc-4f00-ae0d-65c41cd89c2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:03:06.248262 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085c85b0-79cc-4f00-ae0d-65c41cd89c2a service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:03:06.256664 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1417/2822] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 11:03:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 11:03:08.438745 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d143e6b3-8ab8-4bfd-ab76-bbb97d6ab7b1 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:03:08.449034 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d143e6b3-8ab8-4bfd-ab76-bbb97d6ab7b1 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:03:08.453202 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aea1b93-8c85-4f7b-a4bc-f7860fe409ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:03:08.453814 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aea1b93-8c85-4f7b-a4bc-f7860fe409ad service neutron] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:03:08.454192 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aea1b93-8c85-4f7b-a4bc-f7860fe409ad service neutron] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:03:08.454465 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aea1b93-8c85-4f7b-a4bc-f7860fe409ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:03:08.456282 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-5aea1b93-8c85-4f7b-a4bc-f7860fe409ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:03:08.467108 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aea1b93-8c85-4f7b-a4bc-f7860fe409ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:03:08.468602 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aea1b93-8c85-4f7b-a4bc-f7860fe409ad service neutron] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:03:08.478603 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1406/2823] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 11:03:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 11:03:16.252550 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=12 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 11:03:18.473419 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=9 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 11:04:21.946191 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085c85b0-79cc-4f00-ae0d-65c41cd89c2a service neutron] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:04:21.947860 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-085c85b0-79cc-4f00-ae0d-65c41cd89c2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:04:21.963611 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085c85b0-79cc-4f00-ae0d-65c41cd89c2a service neutron] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:04:21.969220 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cfeae9-9da3-43d3-adbc-44fbe477d0e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bd5356829b6454993bab9bad2f79fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:04:21.969820 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cfeae9-9da3-43d3-adbc-44fbe477d0e8 service nova] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:04:21.970103 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cfeae9-9da3-43d3-adbc-44fbe477d0e8 service nova] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:04:21.970376 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cfeae9-9da3-43d3-adbc-44fbe477d0e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:04:21.972069 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-f2cfeae9-9da3-43d3-adbc-44fbe477d0e8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:04:21.983034 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cfeae9-9da3-43d3-adbc-44fbe477d0e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:04:21.984640 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cfeae9-9da3-43d3-adbc-44fbe477d0e8 service nova] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:04:21.993880 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1418/2824] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 11:04:21 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 11:04:31.989764 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=6 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 11:04:38.410056 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361a9ca7-7b18-4d62-a2e0-4df9ba69754d None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:04:38.410611 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361a9ca7-7b18-4d62-a2e0-4df9ba69754d None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:04:38.410611 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361a9ca7-7b18-4d62-a2e0-4df9ba69754d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:04:38.428872 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-361a9ca7-7b18-4d62-a2e0-4df9ba69754d None None] MFA Rules not processed for user `f071b2156180461aa0ee9c4c80acdc68`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 11:04:38.430051 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-361a9ca7-7b18-4d62-a2e0-4df9ba69754d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:04:38.437272 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1407/2825] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 11:04:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 11:04:38.746478 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cfeae9-9da3-43d3-adbc-44fbe477d0e8 service nova] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:04:38.755850 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cfeae9-9da3-43d3-adbc-44fbe477d0e8 service nova] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:04:38.759474 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a854df4-4d0a-41b7-be27-93b9e8d93897 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:04:38.760068 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a854df4-4d0a-41b7-be27-93b9e8d93897 service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:04:38.760348 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a854df4-4d0a-41b7-be27-93b9e8d93897 service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:04:38.760635 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a854df4-4d0a-41b7-be27-93b9e8d93897 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:04:38.762330 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-6a854df4-4d0a-41b7-be27-93b9e8d93897 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:04:38.771825 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a854df4-4d0a-41b7-be27-93b9e8d93897 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:04:38.773312 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a854df4-4d0a-41b7-be27-93b9e8d93897 service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:04:38.782094 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1419/2826] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 11:04:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 11:04:39.101121 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361a9ca7-7b18-4d62-a2e0-4df9ba69754d None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:04:39.102544 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-361a9ca7-7b18-4d62-a2e0-4df9ba69754d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:04:39.173294 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361a9ca7-7b18-4d62-a2e0-4df9ba69754d None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:04:39.178198 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1cb510-b9f5-4e39-810e-0500e234d666 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08477b4c88f04c9e97391aba401879e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:04:39.178794 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1cb510-b9f5-4e39-810e-0500e234d666 service placement] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:04:39.179070 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1cb510-b9f5-4e39-810e-0500e234d666 service placement] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:04:39.179336 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1cb510-b9f5-4e39-810e-0500e234d666 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:04:39.181056 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-af1cb510-b9f5-4e39-810e-0500e234d666 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:04:39.193076 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1cb510-b9f5-4e39-810e-0500e234d666 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:04:39.194624 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1cb510-b9f5-4e39-810e-0500e234d666 service placement] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:04:39.204913 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1408/2827] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 11:04:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 11:04:46.281649 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a854df4-4d0a-41b7-be27-93b9e8d93897 service neutron] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:04:46.296093 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a854df4-4d0a-41b7-be27-93b9e8d93897 service neutron] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:04:46.301943 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a16f53a-0906-4c99-b0dd-fa39fb1c0109 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bd5356829b6454993bab9bad2f79fed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:04:46.302541 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a16f53a-0906-4c99-b0dd-fa39fb1c0109 service nova] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:04:46.302822 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a16f53a-0906-4c99-b0dd-fa39fb1c0109 service nova] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:04:46.303100 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a16f53a-0906-4c99-b0dd-fa39fb1c0109 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:04:46.304888 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-3a16f53a-0906-4c99-b0dd-fa39fb1c0109 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:04:46.318588 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a16f53a-0906-4c99-b0dd-fa39fb1c0109 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:04:46.320386 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a16f53a-0906-4c99-b0dd-fa39fb1c0109 service nova] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:04:46.332082 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1420/2828] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 11:04:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 11:04:49.200103 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=35 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 11:04:56.327209 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=12 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 11:04:57.301731 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1cb510-b9f5-4e39-810e-0500e234d666 service placement] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:04:57.360963 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1cb510-b9f5-4e39-810e-0500e234d666 service placement] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:04:57.366258 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f49d35-d7db-4005-a6ad-42d55b4afa6b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08477b4c88f04c9e97391aba401879e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:04:57.366853 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f49d35-d7db-4005-a6ad-42d55b4afa6b service placement] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:04:57.367135 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f49d35-d7db-4005-a6ad-42d55b4afa6b service placement] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:04:57.367407 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f49d35-d7db-4005-a6ad-42d55b4afa6b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:04:57.369140 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-b9f49d35-d7db-4005-a6ad-42d55b4afa6b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:04:57.403280 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f49d35-d7db-4005-a6ad-42d55b4afa6b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:04:57.404870 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f49d35-d7db-4005-a6ad-42d55b4afa6b service placement] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:04:57.415389 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1409/2829] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 11:04:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 11:05:07.411136 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=36 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 11:06:13.373023 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a16f53a-0906-4c99-b0dd-fa39fb1c0109 service nova] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:13.374492 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-3a16f53a-0906-4c99-b0dd-fa39fb1c0109 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:06:13.435835 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a16f53a-0906-4c99-b0dd-fa39fb1c0109 service nova] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:13.439385 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a0b28f-1d35-4601-870c-5ab5bb3e05a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:13.440015 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a0b28f-1d35-4601-870c-5ab5bb3e05a1 service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:13.440297 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a0b28f-1d35-4601-870c-5ab5bb3e05a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:13.440575 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a0b28f-1d35-4601-870c-5ab5bb3e05a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:13.442319 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-73a0b28f-1d35-4601-870c-5ab5bb3e05a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:06:13.527241 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a0b28f-1d35-4601-870c-5ab5bb3e05a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:13.528607 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a0b28f-1d35-4601-870c-5ab5bb3e05a1 service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:13.548394 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1421/2830] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 11:06:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 11:06:16.538536 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f49d35-d7db-4005-a6ad-42d55b4afa6b service placement] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:16.543613 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-b9f49d35-d7db-4005-a6ad-42d55b4afa6b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:06:16.562777 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f49d35-d7db-4005-a6ad-42d55b4afa6b service placement] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:16.584945 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ee2f86-3cbb-49a2-b9fc-26b4f30bf160 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08477b4c88f04c9e97391aba401879e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:16.585580 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ee2f86-3cbb-49a2-b9fc-26b4f30bf160 service placement] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:16.585859 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ee2f86-3cbb-49a2-b9fc-26b4f30bf160 service placement] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:16.586133 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ee2f86-3cbb-49a2-b9fc-26b4f30bf160 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:16.588112 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-a2ee2f86-3cbb-49a2-b9fc-26b4f30bf160 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:06:16.610794 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ee2f86-3cbb-49a2-b9fc-26b4f30bf160 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:16.615327 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ee2f86-3cbb-49a2-b9fc-26b4f30bf160 service placement] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:16.635294 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a0b28f-1d35-4601-870c-5ab5bb3e05a1 service neutron] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:16.636918 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-73a0b28f-1d35-4601-870c-5ab5bb3e05a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:06:16.646949 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1410/2831] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 11:06:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 11:06:16.649996 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a0b28f-1d35-4601-870c-5ab5bb3e05a1 service neutron] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:16.655678 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4c3fae-6252-45ab-a69b-259de599286b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:16.656664 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4c3fae-6252-45ab-a69b-259de599286b admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:16.656947 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4c3fae-6252-45ab-a69b-259de599286b admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:16.657221 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4c3fae-6252-45ab-a69b-259de599286b admin admin] PATH_INFO: `/v3/users/f071b2156180461aa0ee9c4c80acdc68` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:16.658470 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4c3fae-6252-45ab-a69b-259de599286b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f071b2156180461aa0ee9c4c80acdc68)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:16.661678 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4c3fae-6252-45ab-a69b-259de599286b admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:16.721266 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-4f4c3fae-6252-45ab-a69b-259de599286b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f071b2156180461aa0ee9c4c80acdc68'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:16.729447 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-4f4c3fae-6252-45ab-a69b-259de599286b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f071b2156180461aa0ee9c4c80acdc68'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:16.730200 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-4f4c3fae-6252-45ab-a69b-259de599286b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f071b2156180461aa0ee9c4c80acdc68'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:16.730811 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-4f4c3fae-6252-45ab-a69b-259de599286b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f071b2156180461aa0ee9c4c80acdc68'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:16.744779 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-4f4c3fae-6252-45ab-a69b-259de599286b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f071b2156180461aa0ee9c4c80acdc68'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:16.745943 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 11:06:16.759752 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1422/2832] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 11:06:16 2022] DELETE /identity/v3/users/f071b2156180461aa0ee9c4c80acdc68 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 11:06:16.927642 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ee2f86-3cbb-49a2-b9fc-26b4f30bf160 service placement] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:16.939582 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ee2f86-3cbb-49a2-b9fc-26b4f30bf160 service placement] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:16.996521 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111518ba-8770-47bb-9176-fe07f4392f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:16.997072 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111518ba-8770-47bb-9176-fe07f4392f7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:16.997356 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111518ba-8770-47bb-9176-fe07f4392f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:16.997627 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111518ba-8770-47bb-9176-fe07f4392f7b admin admin] PATH_INFO: `/v3/projects/ecad09476bb3444da0ca3ef5342bb0ef` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:16.998719 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111518ba-8770-47bb-9176-fe07f4392f7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecad09476bb3444da0ca3ef5342bb0ef)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:17.000653 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111518ba-8770-47bb-9176-fe07f4392f7b admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:17.046511 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-111518ba-8770-47bb-9176-fe07f4392f7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecad09476bb3444da0ca3ef5342bb0ef', 'request_id': 'req-111518ba-8770-47bb-9176-fe07f4392f7b'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:17.052625 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-111518ba-8770-47bb-9176-fe07f4392f7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecad09476bb3444da0ca3ef5342bb0ef', 'request_id': 'req-111518ba-8770-47bb-9176-fe07f4392f7b'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:17.060007 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-111518ba-8770-47bb-9176-fe07f4392f7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecad09476bb3444da0ca3ef5342bb0ef', 'request_id': 'req-111518ba-8770-47bb-9176-fe07f4392f7b'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:17.070206 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-111518ba-8770-47bb-9176-fe07f4392f7b admin admin] The token cache is being invalidate because project ecad09476bb3444da0ca3ef5342bb0ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90653) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 11:06:17.070494 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-111518ba-8770-47bb-9176-fe07f4392f7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:17.071126 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-111518ba-8770-47bb-9176-fe07f4392f7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:17.072585 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1411/2833] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 11:06:16 2022] DELETE /identity/v3/projects/ecad09476bb3444da0ca3ef5342bb0ef => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 11:06:17.077567 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298cba0f-bcd6-4634-92e0-436c465f3255 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:17.077860 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298cba0f-bcd6-4634-92e0-436c465f3255 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:17.078142 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298cba0f-bcd6-4634-92e0-436c465f3255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:17.135506 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-298cba0f-bcd6-4634-92e0-436c465f3255 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 11:06:17.136705 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-298cba0f-bcd6-4634-92e0-436c465f3255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:06:17.194338 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1423/2834] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 11:06:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 11:06:17.202980 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b6454-06fa-4d23-a8ad-323e6fb9cec0 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:17.203261 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b6454-06fa-4d23-a8ad-323e6fb9cec0 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:17.203566 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b6454-06fa-4d23-a8ad-323e6fb9cec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:17.224278 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-707b6454-06fa-4d23-a8ad-323e6fb9cec0 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 11:06:17.225470 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-707b6454-06fa-4d23-a8ad-323e6fb9cec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:06:17.233166 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1412/2835] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 11:06:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 11:06:17.237347 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298cba0f-bcd6-4634-92e0-436c465f3255 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:17.247589 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298cba0f-bcd6-4634-92e0-436c465f3255 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:17.252089 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ff4ce0-dc67-4bc9-8482-3cdfbfbc1e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:17.252689 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff4ce0-dc67-4bc9-8482-3cdfbfbc1e4c admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:17.253528 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff4ce0-dc67-4bc9-8482-3cdfbfbc1e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:17.253964 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff4ce0-dc67-4bc9-8482-3cdfbfbc1e4c admin admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:17.254604 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ff4ce0-dc67-4bc9-8482-3cdfbfbc1e4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 11:06:17.254909 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ff4ce0-dc67-4bc9-8482-3cdfbfbc1e4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:17.256545 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ff4ce0-dc67-4bc9-8482-3cdfbfbc1e4c admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:17.267809 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1424/2836] 10.222.0.15 () {58 vars in 1153 bytes} [Sun Sep 11 11:06:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 11:06:17.272549 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707b6454-06fa-4d23-a8ad-323e6fb9cec0 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:17.281843 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707b6454-06fa-4d23-a8ad-323e6fb9cec0 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:17.286916 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c877a0b-08d7-45af-b6e5-6f01bcbf3c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:17.287544 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c877a0b-08d7-45af-b6e5-6f01bcbf3c56 admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:17.287856 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c877a0b-08d7-45af-b6e5-6f01bcbf3c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:17.288146 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c877a0b-08d7-45af-b6e5-6f01bcbf3c56 admin admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:17.288766 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c877a0b-08d7-45af-b6e5-6f01bcbf3c56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:17.290693 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c877a0b-08d7-45af-b6e5-6f01bcbf3c56 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:17.318802 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1413/2837] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 11:06:17 2022] POST /identity/v3/projects => generated 393 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 11:06:17.322931 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ff4ce0-dc67-4bc9-8482-3cdfbfbc1e4c admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:17.333822 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ff4ce0-dc67-4bc9-8482-3cdfbfbc1e4c admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:17.390182 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972d0f39-8c58-48ad-bf5a-c4a1294950ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:17.390786 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972d0f39-8c58-48ad-bf5a-c4a1294950ba admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:17.391059 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972d0f39-8c58-48ad-bf5a-c4a1294950ba admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:17.391326 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972d0f39-8c58-48ad-bf5a-c4a1294950ba admin admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:17.391982 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972d0f39-8c58-48ad-bf5a-c4a1294950ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:17.394969 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972d0f39-8c58-48ad-bf5a-c4a1294950ba admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:17.417017 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1425/2838] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 11:06:17 2022] POST /identity/v3/users => generated 297 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 11:06:17.421138 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c877a0b-08d7-45af-b6e5-6f01bcbf3c56 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:17.431405 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c877a0b-08d7-45af-b6e5-6f01bcbf3c56 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:17.436006 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80a7a6e-ac0d-4118-a453-5e7cab2560d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:17.436596 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a7a6e-ac0d-4118-a453-5e7cab2560d2 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:17.436876 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a7a6e-ac0d-4118-a453-5e7cab2560d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:17.437150 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a7a6e-ac0d-4118-a453-5e7cab2560d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:17.437784 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80a7a6e-ac0d-4118-a453-5e7cab2560d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:17.439370 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80a7a6e-ac0d-4118-a453-5e7cab2560d2 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:17.449719 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1414/2839] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 11:06:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 11:06:17.453586 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972d0f39-8c58-48ad-bf5a-c4a1294950ba admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:17.464700 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972d0f39-8c58-48ad-bf5a-c4a1294950ba admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:17.469256 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0f223f-7249-49e3-9e21-897a900536e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:17.469852 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0f223f-7249-49e3-9e21-897a900536e9 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:17.470132 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0f223f-7249-49e3-9e21-897a900536e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:17.470426 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0f223f-7249-49e3-9e21-897a900536e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:17.471038 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0f223f-7249-49e3-9e21-897a900536e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:17.472699 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0f223f-7249-49e3-9e21-897a900536e9 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:17.483256 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1426/2840] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 11:06:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 11:06:17.487454 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80a7a6e-ac0d-4118-a453-5e7cab2560d2 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:17.497433 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80a7a6e-ac0d-4118-a453-5e7cab2560d2 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:17.501929 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2af34cb-4f07-43be-b087-37af1c98816f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:17.502503 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2af34cb-4f07-43be-b087-37af1c98816f admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:17.502779 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2af34cb-4f07-43be-b087-37af1c98816f admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:17.503649 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2af34cb-4f07-43be-b087-37af1c98816f admin admin] PATH_INFO: `/v3/projects/f3e28efa80524b8fb7578bb30e5313f9/users/565ed04f2c544afbb3a4df1841869d03/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:17.517911 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2af34cb-4f07-43be-b087-37af1c98816f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3e28efa80524b8fb7578bb30e5313f9, user_id=565ed04f2c544afbb3a4df1841869d03, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:17.521304 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2af34cb-4f07-43be-b087-37af1c98816f admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:17.533830 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1415/2841] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 11:06:17 2022] PUT /identity/v3/projects/f3e28efa80524b8fb7578bb30e5313f9/users/565ed04f2c544afbb3a4df1841869d03/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 11:06:17.544088 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8de25-9b23-4d7a-9b09-898aa2dfb888 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:17.544381 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8de25-9b23-4d7a-9b09-898aa2dfb888 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:17.544661 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8de25-9b23-4d7a-9b09-898aa2dfb888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:17.574100 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-7be8de25-9b23-4d7a-9b09-898aa2dfb888 None None] MFA Rules not processed for user `565ed04f2c544afbb3a4df1841869d03`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 11:06:17.575348 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-7be8de25-9b23-4d7a-9b09-898aa2dfb888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:06:17.587618 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 11:06:17.640864 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1427/2842] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 11:06:17 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 11:06:17.645531 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2af34cb-4f07-43be-b087-37af1c98816f admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:17.658887 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 11:06:17.710196 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2af34cb-4f07-43be-b087-37af1c98816f admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:17.714920 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d4fcbb-eb37-4a6b-b472-46e5fada66f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:17.715551 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d4fcbb-eb37-4a6b-b472-46e5fada66f8 admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:17.715832 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d4fcbb-eb37-4a6b-b472-46e5fada66f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:17.716102 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d4fcbb-eb37-4a6b-b472-46e5fada66f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:17.716719 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d4fcbb-eb37-4a6b-b472-46e5fada66f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:17.718620 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d4fcbb-eb37-4a6b-b472-46e5fada66f8 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:17.745598 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1416/2843] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 11:06:17 2022] POST /identity/v3/projects => generated 391 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 11:06:17.749708 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be8de25-9b23-4d7a-9b09-898aa2dfb888 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:17.761336 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be8de25-9b23-4d7a-9b09-898aa2dfb888 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:17.816017 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073437d4-5f93-45a1-a465-766c461fb66a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:17.816645 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073437d4-5f93-45a1-a465-766c461fb66a admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:17.816928 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073437d4-5f93-45a1-a465-766c461fb66a admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:17.817205 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073437d4-5f93-45a1-a465-766c461fb66a admin admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:17.817842 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073437d4-5f93-45a1-a465-766c461fb66a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:17.819789 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073437d4-5f93-45a1-a465-766c461fb66a admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:17.840419 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1428/2844] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 11:06:17 2022] POST /identity/v3/users => generated 302 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 11:06:17.843855 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d4fcbb-eb37-4a6b-b472-46e5fada66f8 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:17.854018 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d4fcbb-eb37-4a6b-b472-46e5fada66f8 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:17.858779 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ad77aa-4b4c-40b3-8b79-a93f82f4db96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:17.859465 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad77aa-4b4c-40b3-8b79-a93f82f4db96 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:17.859756 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad77aa-4b4c-40b3-8b79-a93f82f4db96 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:17.860063 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad77aa-4b4c-40b3-8b79-a93f82f4db96 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:17.860699 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ad77aa-4b4c-40b3-8b79-a93f82f4db96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:17.863042 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ad77aa-4b4c-40b3-8b79-a93f82f4db96 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:17.873478 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1417/2845] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 11:06:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 11:06:17.877519 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073437d4-5f93-45a1-a465-766c461fb66a admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:17.888197 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073437d4-5f93-45a1-a465-766c461fb66a admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:17.892782 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e7628d-70d4-4a10-944d-92aa0bcc8fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:17.893453 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e7628d-70d4-4a10-944d-92aa0bcc8fc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:17.893928 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e7628d-70d4-4a10-944d-92aa0bcc8fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:17.894220 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e7628d-70d4-4a10-944d-92aa0bcc8fc9 admin admin] PATH_INFO: `/v3/projects/79f7c0e1ca5f480c80f5da4a264c8212/users/dd58478b17f7484698315c6ab8ef5faa/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:17.909116 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e7628d-70d4-4a10-944d-92aa0bcc8fc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79f7c0e1ca5f480c80f5da4a264c8212, user_id=dd58478b17f7484698315c6ab8ef5faa, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:17.912169 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e7628d-70d4-4a10-944d-92aa0bcc8fc9 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:17.924532 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1429/2846] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 11:06:17 2022] PUT /identity/v3/projects/79f7c0e1ca5f480c80f5da4a264c8212/users/dd58478b17f7484698315c6ab8ef5faa/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 11:06:17.928993 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ad77aa-4b4c-40b3-8b79-a93f82f4db96 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:17.939295 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ad77aa-4b4c-40b3-8b79-a93f82f4db96 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:17.996653 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92ef8be-8539-43cd-b170-a6ec39ab7348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:17.997254 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ef8be-8539-43cd-b170-a6ec39ab7348 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:17.997533 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ef8be-8539-43cd-b170-a6ec39ab7348 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:17.997804 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ef8be-8539-43cd-b170-a6ec39ab7348 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:17.998531 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92ef8be-8539-43cd-b170-a6ec39ab7348 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:18.000170 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92ef8be-8539-43cd-b170-a6ec39ab7348 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:18.010142 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1418/2847] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 11:06:17 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 11:06:18.014329 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e7628d-70d4-4a10-944d-92aa0bcc8fc9 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:18.024930 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e7628d-70d4-4a10-944d-92aa0bcc8fc9 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:18.029414 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7d2d31-a909-4be3-a436-167c241ffeea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:18.030055 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7d2d31-a909-4be3-a436-167c241ffeea admin admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:18.030332 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7d2d31-a909-4be3-a436-167c241ffeea admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:18.030602 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7d2d31-a909-4be3-a436-167c241ffeea admin admin] PATH_INFO: `/v3/projects/79f7c0e1ca5f480c80f5da4a264c8212/users/dd58478b17f7484698315c6ab8ef5faa/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:18.032521 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7d2d31-a909-4be3-a436-167c241ffeea admin admin] RBAC: Authorizing `identity:create_grant(project_id=79f7c0e1ca5f480c80f5da4a264c8212, user_id=dd58478b17f7484698315c6ab8ef5faa, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:18.036016 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7d2d31-a909-4be3-a436-167c241ffeea admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:18.045887 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1430/2848] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 11:06:18 2022] PUT /identity/v3/projects/79f7c0e1ca5f480c80f5da4a264c8212/users/dd58478b17f7484698315c6ab8ef5faa/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 11:06:18.056719 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca2600-a2a2-4576-95b0-790db04b6ecf None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:18.057012 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca2600-a2a2-4576-95b0-790db04b6ecf None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:18.057295 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca2600-a2a2-4576-95b0-790db04b6ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:18.085087 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-e6ca2600-a2a2-4576-95b0-790db04b6ecf None None] MFA Rules not processed for user `dd58478b17f7484698315c6ab8ef5faa`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 11:06:18.086379 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-e6ca2600-a2a2-4576-95b0-790db04b6ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:06:18.152933 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1419/2849] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 11:06:18 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 11:06:18.164232 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7d2d31-a909-4be3-a436-167c241ffeea admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:18.165912 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-3f7d2d31-a909-4be3-a436-167c241ffeea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:06:18.244129 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 11:06:18.248845 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7d2d31-a909-4be3-a436-167c241ffeea admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:18.252976 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7925fbd-fdc1-42c5-982b-8662182dcb08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:18.253584 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7925fbd-fdc1-42c5-982b-8662182dcb08 service cinder] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:18.253864 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7925fbd-fdc1-42c5-982b-8662182dcb08 service cinder] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:18.254137 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7925fbd-fdc1-42c5-982b-8662182dcb08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:18.262974 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7925fbd-fdc1-42c5-982b-8662182dcb08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:18.264501 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7925fbd-fdc1-42c5-982b-8662182dcb08 service cinder] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:18.273824 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1431/2850] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 11:06:18 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 11:06:18.320726 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca2600-a2a2-4576-95b0-790db04b6ecf None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:18.322185 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-e6ca2600-a2a2-4576-95b0-790db04b6ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:06:18.334250 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca2600-a2a2-4576-95b0-790db04b6ecf None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:18.337922 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4892af8d-33d1-4225-9707-7b2f65db198e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:18.338512 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4892af8d-33d1-4225-9707-7b2f65db198e service cinder] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:18.338791 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4892af8d-33d1-4225-9707-7b2f65db198e service cinder] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:18.339062 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4892af8d-33d1-4225-9707-7b2f65db198e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:18.344302 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 11:06:18.398443 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4892af8d-33d1-4225-9707-7b2f65db198e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:18.399893 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4892af8d-33d1-4225-9707-7b2f65db198e service cinder] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:18.408686 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1420/2851] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 11:06:18 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 11:06:21.682618 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7925fbd-fdc1-42c5-982b-8662182dcb08 service cinder] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:21.745579 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7925fbd-fdc1-42c5-982b-8662182dcb08 service cinder] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:21.750394 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b97abee-2225-43c0-9b6e-a82f8e06ebf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:21.751173 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b97abee-2225-43c0-9b6e-a82f8e06ebf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:21.751523 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b97abee-2225-43c0-9b6e-a82f8e06ebf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:21.751801 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b97abee-2225-43c0-9b6e-a82f8e06ebf8 admin admin] PATH_INFO: `/v3/users/565ed04f2c544afbb3a4df1841869d03` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:21.752941 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b97abee-2225-43c0-9b6e-a82f8e06ebf8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=565ed04f2c544afbb3a4df1841869d03)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:21.754910 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b97abee-2225-43c0-9b6e-a82f8e06ebf8 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:21.799130 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-6b97abee-2225-43c0-9b6e-a82f8e06ebf8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '565ed04f2c544afbb3a4df1841869d03'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:21.806860 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-6b97abee-2225-43c0-9b6e-a82f8e06ebf8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '565ed04f2c544afbb3a4df1841869d03'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:21.807499 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-6b97abee-2225-43c0-9b6e-a82f8e06ebf8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '565ed04f2c544afbb3a4df1841869d03'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:21.808112 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-6b97abee-2225-43c0-9b6e-a82f8e06ebf8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '565ed04f2c544afbb3a4df1841869d03'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:21.819950 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-6b97abee-2225-43c0-9b6e-a82f8e06ebf8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '565ed04f2c544afbb3a4df1841869d03'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:21.831935 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1432/2852] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 11:06:21 2022] DELETE /identity/v3/users/565ed04f2c544afbb3a4df1841869d03 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 11:06:21.836485 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46826b3-80fa-4f43-a4ac-ed47bb8cefe3 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:21.836969 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46826b3-80fa-4f43-a4ac-ed47bb8cefe3 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:21.837094 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46826b3-80fa-4f43-a4ac-ed47bb8cefe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:21.857206 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-d46826b3-80fa-4f43-a4ac-ed47bb8cefe3 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 11:06:21.858463 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-d46826b3-80fa-4f43-a4ac-ed47bb8cefe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:06:21.916674 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1421/2853] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 11:06:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 11:06:21.927012 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b97abee-2225-43c0-9b6e-a82f8e06ebf8 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:21.928410 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-6b97abee-2225-43c0-9b6e-a82f8e06ebf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:06:21.983569 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b97abee-2225-43c0-9b6e-a82f8e06ebf8 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:21.987110 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836246a8-1dcb-464d-87f8-56e64fbf1458 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:21.987738 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836246a8-1dcb-464d-87f8-56e64fbf1458 service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:21.988922 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836246a8-1dcb-464d-87f8-56e64fbf1458 service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:21.989197 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836246a8-1dcb-464d-87f8-56e64fbf1458 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:21.998065 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836246a8-1dcb-464d-87f8-56e64fbf1458 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:21.999512 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836246a8-1dcb-464d-87f8-56e64fbf1458 service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:22.009015 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1433/2854] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 11:06:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 11:06:22.273793 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46826b3-80fa-4f43-a4ac-ed47bb8cefe3 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:22.275202 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-d46826b3-80fa-4f43-a4ac-ed47bb8cefe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:06:22.288009 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46826b3-80fa-4f43-a4ac-ed47bb8cefe3 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:22.293208 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ef3902-29ac-4f6c-bde7-c35dd34d788d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:22.293846 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ef3902-29ac-4f6c-bde7-c35dd34d788d admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:22.294141 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ef3902-29ac-4f6c-bde7-c35dd34d788d admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:22.294403 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ef3902-29ac-4f6c-bde7-c35dd34d788d admin admin] PATH_INFO: `/v3/projects/f3e28efa80524b8fb7578bb30e5313f9` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:22.295572 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ef3902-29ac-4f6c-bde7-c35dd34d788d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3e28efa80524b8fb7578bb30e5313f9)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:22.297494 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ef3902-29ac-4f6c-bde7-c35dd34d788d admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:22.350499 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-b7ef3902-29ac-4f6c-bde7-c35dd34d788d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3e28efa80524b8fb7578bb30e5313f9', 'request_id': 'req-b7ef3902-29ac-4f6c-bde7-c35dd34d788d'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:22.356721 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-b7ef3902-29ac-4f6c-bde7-c35dd34d788d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3e28efa80524b8fb7578bb30e5313f9', 'request_id': 'req-b7ef3902-29ac-4f6c-bde7-c35dd34d788d'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:22.364351 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-b7ef3902-29ac-4f6c-bde7-c35dd34d788d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3e28efa80524b8fb7578bb30e5313f9', 'request_id': 'req-b7ef3902-29ac-4f6c-bde7-c35dd34d788d'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:22.374675 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-b7ef3902-29ac-4f6c-bde7-c35dd34d788d admin admin] The token cache is being invalidate because project f3e28efa80524b8fb7578bb30e5313f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90653) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 11:06:22.374964 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-b7ef3902-29ac-4f6c-bde7-c35dd34d788d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:22.375622 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-b7ef3902-29ac-4f6c-bde7-c35dd34d788d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:22.377137 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1422/2855] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 11:06:22 2022] DELETE /identity/v3/projects/f3e28efa80524b8fb7578bb30e5313f9 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 11:06:22.381206 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836246a8-1dcb-464d-87f8-56e64fbf1458 service neutron] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:22.390800 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836246a8-1dcb-464d-87f8-56e64fbf1458 service neutron] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:22.399114 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 11:06:22.448139 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d4e408-bd9a-4b54-bc54-cad805c8db25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:22.448689 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d4e408-bd9a-4b54-bc54-cad805c8db25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:22.448970 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d4e408-bd9a-4b54-bc54-cad805c8db25 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:22.449245 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d4e408-bd9a-4b54-bc54-cad805c8db25 admin admin] PATH_INFO: `/v3/users/dd58478b17f7484698315c6ab8ef5faa` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:22.450363 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d4e408-bd9a-4b54-bc54-cad805c8db25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd58478b17f7484698315c6ab8ef5faa)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:22.452306 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d4e408-bd9a-4b54-bc54-cad805c8db25 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:22.498124 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-a9d4e408-bd9a-4b54-bc54-cad805c8db25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd58478b17f7484698315c6ab8ef5faa'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:22.506013 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-a9d4e408-bd9a-4b54-bc54-cad805c8db25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd58478b17f7484698315c6ab8ef5faa'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:22.506643 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-a9d4e408-bd9a-4b54-bc54-cad805c8db25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd58478b17f7484698315c6ab8ef5faa'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:22.507259 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-a9d4e408-bd9a-4b54-bc54-cad805c8db25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd58478b17f7484698315c6ab8ef5faa'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:22.519810 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-a9d4e408-bd9a-4b54-bc54-cad805c8db25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd58478b17f7484698315c6ab8ef5faa'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:22.531839 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1434/2856] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 11:06:22 2022] DELETE /identity/v3/users/dd58478b17f7484698315c6ab8ef5faa => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 11:06:22.801149 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ef3902-29ac-4f6c-bde7-c35dd34d788d admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:22.802588 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-b7ef3902-29ac-4f6c-bde7-c35dd34d788d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:06:22.817906 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ef3902-29ac-4f6c-bde7-c35dd34d788d admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:22.848635 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 11:06:22.879330 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d839e3-3118-4c0f-b90a-30859167b8c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:22.879995 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d839e3-3118-4c0f-b90a-30859167b8c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:22.880283 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d839e3-3118-4c0f-b90a-30859167b8c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:22.880559 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d839e3-3118-4c0f-b90a-30859167b8c4 admin admin] PATH_INFO: `/v3/projects/79f7c0e1ca5f480c80f5da4a264c8212` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:22.881680 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d839e3-3118-4c0f-b90a-30859167b8c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79f7c0e1ca5f480c80f5da4a264c8212)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:22.883618 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d839e3-3118-4c0f-b90a-30859167b8c4 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:22.929073 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-70d839e3-3118-4c0f-b90a-30859167b8c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79f7c0e1ca5f480c80f5da4a264c8212', 'request_id': 'req-70d839e3-3118-4c0f-b90a-30859167b8c4'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:22.935179 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-70d839e3-3118-4c0f-b90a-30859167b8c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79f7c0e1ca5f480c80f5da4a264c8212', 'request_id': 'req-70d839e3-3118-4c0f-b90a-30859167b8c4'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:22.943253 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-70d839e3-3118-4c0f-b90a-30859167b8c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79f7c0e1ca5f480c80f5da4a264c8212', 'request_id': 'req-70d839e3-3118-4c0f-b90a-30859167b8c4'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:22.953344 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-70d839e3-3118-4c0f-b90a-30859167b8c4 admin admin] The token cache is being invalidate because project 79f7c0e1ca5f480c80f5da4a264c8212 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90653) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 11:06:22.953637 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-70d839e3-3118-4c0f-b90a-30859167b8c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:22.954261 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-70d839e3-3118-4c0f-b90a-30859167b8c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:22.955752 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1423/2857] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 11:06:22 2022] DELETE /identity/v3/projects/79f7c0e1ca5f480c80f5da4a264c8212 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 11:06:22.961113 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f21801-2eaa-42e7-ab13-f013bffcc30b None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:22.961649 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f21801-2eaa-42e7-ab13-f013bffcc30b None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:22.961649 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f21801-2eaa-42e7-ab13-f013bffcc30b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:22.981820 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-d2f21801-2eaa-42e7-ab13-f013bffcc30b None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 11:06:22.982963 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-d2f21801-2eaa-42e7-ab13-f013bffcc30b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:06:23.040950 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1435/2858] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 11:06:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 11:06:23.048900 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102eb7d8-6f38-44d8-84c8-6c108b5d1814 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:23.049181 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102eb7d8-6f38-44d8-84c8-6c108b5d1814 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:23.049502 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102eb7d8-6f38-44d8-84c8-6c108b5d1814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:23.069358 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-102eb7d8-6f38-44d8-84c8-6c108b5d1814 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 11:06:23.070566 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-102eb7d8-6f38-44d8-84c8-6c108b5d1814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:06:23.078393 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1424/2859] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 11:06:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 11:06:23.083367 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f21801-2eaa-42e7-ab13-f013bffcc30b None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:23.093229 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f21801-2eaa-42e7-ab13-f013bffcc30b None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:23.098621 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125b4797-84b7-492d-90ed-add90a9d4cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:23.099229 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b4797-84b7-492d-90ed-add90a9d4cbe admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:23.099593 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b4797-84b7-492d-90ed-add90a9d4cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:23.099870 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b4797-84b7-492d-90ed-add90a9d4cbe admin admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:23.100514 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125b4797-84b7-492d-90ed-add90a9d4cbe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 11:06:23.100815 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125b4797-84b7-492d-90ed-add90a9d4cbe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:23.102405 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125b4797-84b7-492d-90ed-add90a9d4cbe admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:23.114553 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1436/2860] 10.222.0.15 () {58 vars in 1153 bytes} [Sun Sep 11 11:06:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 11:06:23.119301 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102eb7d8-6f38-44d8-84c8-6c108b5d1814 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:23.128683 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102eb7d8-6f38-44d8-84c8-6c108b5d1814 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:23.133316 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daa66de-fab7-4626-81af-410d203ec2e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:23.133915 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daa66de-fab7-4626-81af-410d203ec2e3 admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:23.134190 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daa66de-fab7-4626-81af-410d203ec2e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:23.134457 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daa66de-fab7-4626-81af-410d203ec2e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:23.135081 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daa66de-fab7-4626-81af-410d203ec2e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:23.137136 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daa66de-fab7-4626-81af-410d203ec2e3 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:23.164708 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1425/2861] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 11:06:23 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 11:06:23.169051 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125b4797-84b7-492d-90ed-add90a9d4cbe admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:23.179069 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125b4797-84b7-492d-90ed-add90a9d4cbe admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:23.191223 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 11:06:23.236798 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75804a5e-a35f-4477-96e7-4145aa37333e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:23.237327 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75804a5e-a35f-4477-96e7-4145aa37333e admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:23.237613 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75804a5e-a35f-4477-96e7-4145aa37333e admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:23.238191 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75804a5e-a35f-4477-96e7-4145aa37333e admin admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:23.238914 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75804a5e-a35f-4477-96e7-4145aa37333e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:23.240864 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75804a5e-a35f-4477-96e7-4145aa37333e admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:23.261754 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1437/2862] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 11:06:23 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 11:06:23.265965 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daa66de-fab7-4626-81af-410d203ec2e3 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:23.277006 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daa66de-fab7-4626-81af-410d203ec2e3 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:23.281582 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304ccbc2-437d-45eb-94a0-3f3e53554001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:23.282167 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304ccbc2-437d-45eb-94a0-3f3e53554001 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:23.282448 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304ccbc2-437d-45eb-94a0-3f3e53554001 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:23.282719 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304ccbc2-437d-45eb-94a0-3f3e53554001 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:23.283348 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304ccbc2-437d-45eb-94a0-3f3e53554001 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:23.285180 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304ccbc2-437d-45eb-94a0-3f3e53554001 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:23.295206 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1426/2863] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 11:06:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 11:06:23.299216 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75804a5e-a35f-4477-96e7-4145aa37333e admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:23.309392 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75804a5e-a35f-4477-96e7-4145aa37333e admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:23.314276 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b89c3f-990b-4bb5-b201-332e7f72860a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:23.314911 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b89c3f-990b-4bb5-b201-332e7f72860a admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:23.315222 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b89c3f-990b-4bb5-b201-332e7f72860a admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:23.315530 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b89c3f-990b-4bb5-b201-332e7f72860a admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:23.316729 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b89c3f-990b-4bb5-b201-332e7f72860a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:23.318379 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b89c3f-990b-4bb5-b201-332e7f72860a admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:23.334069 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1438/2864] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 11:06:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 11:06:23.338386 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304ccbc2-437d-45eb-94a0-3f3e53554001 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:23.348837 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304ccbc2-437d-45eb-94a0-3f3e53554001 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:23.353370 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca03e76-9dc4-4370-841e-ad6b703b2faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:23.354010 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca03e76-9dc4-4370-841e-ad6b703b2faa admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:23.354287 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca03e76-9dc4-4370-841e-ad6b703b2faa admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:23.354555 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca03e76-9dc4-4370-841e-ad6b703b2faa admin admin] PATH_INFO: `/v3/projects/76ff542e0c85460ba43460282bebeccb/users/cd1fbcdc3146487ba416764cb5bcd622/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:23.369480 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca03e76-9dc4-4370-841e-ad6b703b2faa admin admin] RBAC: Authorizing `identity:create_grant(project_id=76ff542e0c85460ba43460282bebeccb, user_id=cd1fbcdc3146487ba416764cb5bcd622, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:23.372142 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca03e76-9dc4-4370-841e-ad6b703b2faa admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:23.385494 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1427/2865] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 11:06:23 2022] PUT /identity/v3/projects/76ff542e0c85460ba43460282bebeccb/users/cd1fbcdc3146487ba416764cb5bcd622/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 11:06:23.396757 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a18871-97c6-4fb7-9308-72b9bce567ce None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:23.397070 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a18871-97c6-4fb7-9308-72b9bce567ce None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:23.397315 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a18871-97c6-4fb7-9308-72b9bce567ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:23.424167 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-39a18871-97c6-4fb7-9308-72b9bce567ce None None] MFA Rules not processed for user `cd1fbcdc3146487ba416764cb5bcd622`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 11:06:23.425380 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-39a18871-97c6-4fb7-9308-72b9bce567ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:06:23.489899 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1439/2866] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 11:06:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 11:06:23.494435 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca03e76-9dc4-4370-841e-ad6b703b2faa admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:23.551511 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 11:06:23.555501 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca03e76-9dc4-4370-841e-ad6b703b2faa admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:23.561451 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1e34bc-f9d3-4680-8f92-d43f795636d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:23.562291 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1e34bc-f9d3-4680-8f92-d43f795636d5 admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:23.562586 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1e34bc-f9d3-4680-8f92-d43f795636d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:23.562966 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1e34bc-f9d3-4680-8f92-d43f795636d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:23.563682 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1e34bc-f9d3-4680-8f92-d43f795636d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:23.565725 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1e34bc-f9d3-4680-8f92-d43f795636d5 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:23.592182 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1428/2867] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 11:06:23 2022] POST /identity/v3/projects => generated 377 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 11:06:23.596319 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a18871-97c6-4fb7-9308-72b9bce567ce None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:23.607102 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a18871-97c6-4fb7-9308-72b9bce567ce None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:23.661980 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd3b1bf-1779-4ec9-824c-3191ae868e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:23.662783 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd3b1bf-1779-4ec9-824c-3191ae868e76 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:23.663071 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd3b1bf-1779-4ec9-824c-3191ae868e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:23.663445 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd3b1bf-1779-4ec9-824c-3191ae868e76 admin admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:23.664083 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd3b1bf-1779-4ec9-824c-3191ae868e76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:23.666031 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd3b1bf-1779-4ec9-824c-3191ae868e76 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:23.686806 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1440/2868] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 11:06:23 2022] POST /identity/v3/users => generated 289 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 11:06:23.690770 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1e34bc-f9d3-4680-8f92-d43f795636d5 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:23.701509 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1e34bc-f9d3-4680-8f92-d43f795636d5 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:23.706083 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847acbd4-349e-4404-bdbb-6c03d10c57f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:23.706684 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847acbd4-349e-4404-bdbb-6c03d10c57f9 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:23.706962 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847acbd4-349e-4404-bdbb-6c03d10c57f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:23.707237 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847acbd4-349e-4404-bdbb-6c03d10c57f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:23.707910 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847acbd4-349e-4404-bdbb-6c03d10c57f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:23.709539 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847acbd4-349e-4404-bdbb-6c03d10c57f9 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:23.719895 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1429/2869] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 11:06:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 11:06:23.723792 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd3b1bf-1779-4ec9-824c-3191ae868e76 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:23.733445 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd3b1bf-1779-4ec9-824c-3191ae868e76 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:23.738478 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fc7ff2-eab4-4dc1-adba-d4d14699d7d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:23.739062 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fc7ff2-eab4-4dc1-adba-d4d14699d7d3 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:23.739336 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fc7ff2-eab4-4dc1-adba-d4d14699d7d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:23.739634 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fc7ff2-eab4-4dc1-adba-d4d14699d7d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:23.740351 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fc7ff2-eab4-4dc1-adba-d4d14699d7d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:23.741865 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fc7ff2-eab4-4dc1-adba-d4d14699d7d3 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:23.751618 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1441/2870] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 11:06:23 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 11:06:23.756286 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847acbd4-349e-4404-bdbb-6c03d10c57f9 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:23.766131 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847acbd4-349e-4404-bdbb-6c03d10c57f9 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:23.770984 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005626c2-3947-424e-b9d3-01d77247937e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:23.771664 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005626c2-3947-424e-b9d3-01d77247937e admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:23.771951 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005626c2-3947-424e-b9d3-01d77247937e admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:23.772238 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005626c2-3947-424e-b9d3-01d77247937e admin admin] PATH_INFO: `/v3/projects/83349722398d4170821d3c860ac8cf18/users/04d12a097ea247cbb0242c8d346a7117/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:23.787268 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005626c2-3947-424e-b9d3-01d77247937e admin admin] RBAC: Authorizing `identity:create_grant(project_id=83349722398d4170821d3c860ac8cf18, user_id=04d12a097ea247cbb0242c8d346a7117, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:23.789954 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005626c2-3947-424e-b9d3-01d77247937e admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:23.803161 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1430/2871] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 11:06:23 2022] PUT /identity/v3/projects/83349722398d4170821d3c860ac8cf18/users/04d12a097ea247cbb0242c8d346a7117/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 11:06:23.866792 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2978af7d-151e-4383-b7d4-2075da2e4241 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:23.867074 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2978af7d-151e-4383-b7d4-2075da2e4241 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:23.867382 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2978af7d-151e-4383-b7d4-2075da2e4241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:23.875554 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 11:06:23.897773 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-2978af7d-151e-4383-b7d4-2075da2e4241 None None] MFA Rules not processed for user `04d12a097ea247cbb0242c8d346a7117`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 11:06:23.899012 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-2978af7d-151e-4383-b7d4-2075da2e4241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:06:23.961913 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1442/2872] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 11:06:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 11:06:23.966788 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005626c2-3947-424e-b9d3-01d77247937e admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:24.028770 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005626c2-3947-424e-b9d3-01d77247937e admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:24.033352 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce39ff3e-43a3-4d6c-af10-28521a5f71e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:24.033945 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39ff3e-43a3-4d6c-af10-28521a5f71e1 admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:24.034224 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39ff3e-43a3-4d6c-af10-28521a5f71e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:24.034494 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39ff3e-43a3-4d6c-af10-28521a5f71e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:24.035878 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce39ff3e-43a3-4d6c-af10-28521a5f71e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:24.037770 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce39ff3e-43a3-4d6c-af10-28521a5f71e1 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:24.064079 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1431/2873] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 11:06:23 2022] POST /identity/v3/projects => generated 379 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 11:06:24.068191 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2978af7d-151e-4383-b7d4-2075da2e4241 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:24.078986 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2978af7d-151e-4383-b7d4-2075da2e4241 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:24.134913 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00cbbcb-9856-4c94-a53a-5c1d3103ca30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:24.135545 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00cbbcb-9856-4c94-a53a-5c1d3103ca30 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:24.135825 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00cbbcb-9856-4c94-a53a-5c1d3103ca30 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:24.136097 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00cbbcb-9856-4c94-a53a-5c1d3103ca30 admin admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:24.136712 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00cbbcb-9856-4c94-a53a-5c1d3103ca30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:24.138638 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00cbbcb-9856-4c94-a53a-5c1d3103ca30 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:24.159085 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1443/2874] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 11:06:24 2022] POST /identity/v3/users => generated 296 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 11:06:24.163884 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce39ff3e-43a3-4d6c-af10-28521a5f71e1 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:24.174293 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce39ff3e-43a3-4d6c-af10-28521a5f71e1 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:24.179551 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d85587a-60ee-444b-b4d6-75d0cc2865d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:24.180174 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d85587a-60ee-444b-b4d6-75d0cc2865d1 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:24.180454 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d85587a-60ee-444b-b4d6-75d0cc2865d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:24.180744 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d85587a-60ee-444b-b4d6-75d0cc2865d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:24.181377 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d85587a-60ee-444b-b4d6-75d0cc2865d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:24.182955 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d85587a-60ee-444b-b4d6-75d0cc2865d1 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:24.194561 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1432/2875] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 11:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 11:06:24.198535 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00cbbcb-9856-4c94-a53a-5c1d3103ca30 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:24.208624 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00cbbcb-9856-4c94-a53a-5c1d3103ca30 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:24.213269 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aad4187-2439-4edc-9cc7-8d6e48a2325d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:24.213919 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aad4187-2439-4edc-9cc7-8d6e48a2325d admin admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:24.214202 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aad4187-2439-4edc-9cc7-8d6e48a2325d admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:24.214479 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aad4187-2439-4edc-9cc7-8d6e48a2325d admin admin] PATH_INFO: `/v3/projects/3639b1facaa8495897123ced2e4e101e/users/d7a88ce9806e4e2cb6a87be4c6712d96/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:24.229446 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aad4187-2439-4edc-9cc7-8d6e48a2325d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3639b1facaa8495897123ced2e4e101e, user_id=d7a88ce9806e4e2cb6a87be4c6712d96, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:24.232125 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aad4187-2439-4edc-9cc7-8d6e48a2325d admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:24.245716 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1444/2876] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 11:06:24 2022] PUT /identity/v3/projects/3639b1facaa8495897123ced2e4e101e/users/d7a88ce9806e4e2cb6a87be4c6712d96/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 11:06:24.249613 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d85587a-60ee-444b-b4d6-75d0cc2865d1 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:24.259605 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d85587a-60ee-444b-b4d6-75d0cc2865d1 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:24.315775 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3589b1d7-5a4d-4bab-b703-5f2fea16dda0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:24.316438 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3589b1d7-5a4d-4bab-b703-5f2fea16dda0 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:24.316711 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3589b1d7-5a4d-4bab-b703-5f2fea16dda0 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:24.316976 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3589b1d7-5a4d-4bab-b703-5f2fea16dda0 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:24.317607 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3589b1d7-5a4d-4bab-b703-5f2fea16dda0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:24.319203 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3589b1d7-5a4d-4bab-b703-5f2fea16dda0 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:24.328993 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1433/2877] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 11:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 11:06:24.333582 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aad4187-2439-4edc-9cc7-8d6e48a2325d admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:24.343490 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aad4187-2439-4edc-9cc7-8d6e48a2325d admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:24.348745 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeba1c2-e4de-4d57-babb-400bb1c93cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:24.349384 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeba1c2-e4de-4d57-babb-400bb1c93cda admin admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:24.349661 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeba1c2-e4de-4d57-babb-400bb1c93cda admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:24.349929 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeba1c2-e4de-4d57-babb-400bb1c93cda admin admin] PATH_INFO: `/v3/projects/3639b1facaa8495897123ced2e4e101e/users/d7a88ce9806e4e2cb6a87be4c6712d96/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:24.351820 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeba1c2-e4de-4d57-babb-400bb1c93cda admin admin] RBAC: Authorizing `identity:create_grant(project_id=3639b1facaa8495897123ced2e4e101e, user_id=d7a88ce9806e4e2cb6a87be4c6712d96, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:24.354432 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeba1c2-e4de-4d57-babb-400bb1c93cda admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:24.364870 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1445/2878] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 11:06:24 2022] PUT /identity/v3/projects/3639b1facaa8495897123ced2e4e101e/users/d7a88ce9806e4e2cb6a87be4c6712d96/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 11:06:24.375189 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645cd33f-f6e1-435c-9915-f0eb6eb97e03 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:24.375514 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645cd33f-f6e1-435c-9915-f0eb6eb97e03 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:24.375794 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645cd33f-f6e1-435c-9915-f0eb6eb97e03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:24.403692 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-645cd33f-f6e1-435c-9915-f0eb6eb97e03 None None] MFA Rules not processed for user `d7a88ce9806e4e2cb6a87be4c6712d96`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 11:06:24.404921 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-645cd33f-f6e1-435c-9915-f0eb6eb97e03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:06:24.410534 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 11:06:24.475813 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1434/2879] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 11:06:24 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 11:06:24.486918 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeba1c2-e4de-4d57-babb-400bb1c93cda admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:24.489048 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-cbeba1c2-e4de-4d57-babb-400bb1c93cda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:06:24.542025 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 11:06:24.548049 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeba1c2-e4de-4d57-babb-400bb1c93cda admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:24.552084 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941440ea-7465-4bb8-b29e-f4473992be67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:24.552682 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941440ea-7465-4bb8-b29e-f4473992be67 service cinder] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:24.552957 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941440ea-7465-4bb8-b29e-f4473992be67 service cinder] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:24.553226 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941440ea-7465-4bb8-b29e-f4473992be67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:24.562319 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941440ea-7465-4bb8-b29e-f4473992be67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:24.563721 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941440ea-7465-4bb8-b29e-f4473992be67 service cinder] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:24.573202 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1446/2880] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 11:06:24 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 11:06:24.616038 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645cd33f-f6e1-435c-9915-f0eb6eb97e03 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:24.617467 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-645cd33f-f6e1-435c-9915-f0eb6eb97e03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:06:24.629571 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645cd33f-f6e1-435c-9915-f0eb6eb97e03 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:24.633188 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa464ba-aaf1-4e29-9d62-7460647309de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:24.633780 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa464ba-aaf1-4e29-9d62-7460647309de service cinder] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:24.634058 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa464ba-aaf1-4e29-9d62-7460647309de service cinder] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:24.634329 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa464ba-aaf1-4e29-9d62-7460647309de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:24.689834 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa464ba-aaf1-4e29-9d62-7460647309de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:24.691530 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa464ba-aaf1-4e29-9d62-7460647309de service cinder] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:24.700225 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1435/2881] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 11:06:24 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 11:06:27.604264 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941440ea-7465-4bb8-b29e-f4473992be67 service cinder] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:27.666937 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941440ea-7465-4bb8-b29e-f4473992be67 service cinder] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:27.671460 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3300c67e-882d-49bf-92d1-318bab8ecbae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:27.672081 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3300c67e-882d-49bf-92d1-318bab8ecbae admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:27.672358 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3300c67e-882d-49bf-92d1-318bab8ecbae admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:27.672857 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3300c67e-882d-49bf-92d1-318bab8ecbae admin admin] PATH_INFO: `/v3/users/cd1fbcdc3146487ba416764cb5bcd622` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:27.673950 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3300c67e-882d-49bf-92d1-318bab8ecbae admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd1fbcdc3146487ba416764cb5bcd622)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:27.676432 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3300c67e-882d-49bf-92d1-318bab8ecbae admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:27.720366 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-3300c67e-882d-49bf-92d1-318bab8ecbae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd1fbcdc3146487ba416764cb5bcd622'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:27.728194 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-3300c67e-882d-49bf-92d1-318bab8ecbae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd1fbcdc3146487ba416764cb5bcd622'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:27.728812 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-3300c67e-882d-49bf-92d1-318bab8ecbae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd1fbcdc3146487ba416764cb5bcd622'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:27.729434 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-3300c67e-882d-49bf-92d1-318bab8ecbae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd1fbcdc3146487ba416764cb5bcd622'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:27.741952 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-3300c67e-882d-49bf-92d1-318bab8ecbae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd1fbcdc3146487ba416764cb5bcd622'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:27.754080 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1447/2882] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 11:06:27 2022] DELETE /identity/v3/users/cd1fbcdc3146487ba416764cb5bcd622 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 11:06:27.758728 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd1ea2f-38f7-4552-a9e8-7d24c7084938 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:27.759233 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd1ea2f-38f7-4552-a9e8-7d24c7084938 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:27.759354 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd1ea2f-38f7-4552-a9e8-7d24c7084938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:27.778125 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-ecd1ea2f-38f7-4552-a9e8-7d24c7084938 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 11:06:27.779331 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-ecd1ea2f-38f7-4552-a9e8-7d24c7084938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:06:27.840981 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1436/2883] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 11:06:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 11:06:27.851151 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3300c67e-882d-49bf-92d1-318bab8ecbae admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:27.852577 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-3300c67e-882d-49bf-92d1-318bab8ecbae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:06:27.909110 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3300c67e-882d-49bf-92d1-318bab8ecbae admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:27.913430 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cb36e5-3d75-455d-a233-3f8416db7285 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:27.914041 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cb36e5-3d75-455d-a233-3f8416db7285 service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:27.914322 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cb36e5-3d75-455d-a233-3f8416db7285 service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:27.914595 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cb36e5-3d75-455d-a233-3f8416db7285 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:27.923861 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cb36e5-3d75-455d-a233-3f8416db7285 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:27.925972 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cb36e5-3d75-455d-a233-3f8416db7285 service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:27.935543 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1448/2884] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 11:06:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 11:06:28.198145 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd1ea2f-38f7-4552-a9e8-7d24c7084938 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:28.199894 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-ecd1ea2f-38f7-4552-a9e8-7d24c7084938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:06:28.214106 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd1ea2f-38f7-4552-a9e8-7d24c7084938 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:28.218967 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b264ba52-c5d5-4ee6-88a2-ba830e763932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:28.219765 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b264ba52-c5d5-4ee6-88a2-ba830e763932 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:28.220042 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b264ba52-c5d5-4ee6-88a2-ba830e763932 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:28.220309 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b264ba52-c5d5-4ee6-88a2-ba830e763932 admin admin] PATH_INFO: `/v3/projects/76ff542e0c85460ba43460282bebeccb` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:28.221462 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b264ba52-c5d5-4ee6-88a2-ba830e763932 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76ff542e0c85460ba43460282bebeccb)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:28.224331 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b264ba52-c5d5-4ee6-88a2-ba830e763932 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:28.262308 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 11:06:28.271905 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-b264ba52-c5d5-4ee6-88a2-ba830e763932 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76ff542e0c85460ba43460282bebeccb', 'request_id': 'req-b264ba52-c5d5-4ee6-88a2-ba830e763932'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:28.278171 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-b264ba52-c5d5-4ee6-88a2-ba830e763932 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76ff542e0c85460ba43460282bebeccb', 'request_id': 'req-b264ba52-c5d5-4ee6-88a2-ba830e763932'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:28.285960 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-b264ba52-c5d5-4ee6-88a2-ba830e763932 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76ff542e0c85460ba43460282bebeccb', 'request_id': 'req-b264ba52-c5d5-4ee6-88a2-ba830e763932'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:28.296598 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-b264ba52-c5d5-4ee6-88a2-ba830e763932 admin admin] The token cache is being invalidate because project 76ff542e0c85460ba43460282bebeccb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90653) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 11:06:28.296888 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-b264ba52-c5d5-4ee6-88a2-ba830e763932 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:28.297508 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-b264ba52-c5d5-4ee6-88a2-ba830e763932 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:28.299012 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1437/2885] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 11:06:28 2022] DELETE /identity/v3/projects/76ff542e0c85460ba43460282bebeccb => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 11:06:28.303707 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cb36e5-3d75-455d-a233-3f8416db7285 service neutron] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:28.313218 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cb36e5-3d75-455d-a233-3f8416db7285 service neutron] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:28.321485 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 11:06:28.371084 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2c3174-f474-412d-9502-d8bd3dd63a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:28.371754 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2c3174-f474-412d-9502-d8bd3dd63a17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:28.372034 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2c3174-f474-412d-9502-d8bd3dd63a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:28.372305 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2c3174-f474-412d-9502-d8bd3dd63a17 admin admin] PATH_INFO: `/v3/users/04d12a097ea247cbb0242c8d346a7117` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:28.373411 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2c3174-f474-412d-9502-d8bd3dd63a17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04d12a097ea247cbb0242c8d346a7117)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:28.375376 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2c3174-f474-412d-9502-d8bd3dd63a17 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:28.420612 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-dd2c3174-f474-412d-9502-d8bd3dd63a17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04d12a097ea247cbb0242c8d346a7117'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:28.429280 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-dd2c3174-f474-412d-9502-d8bd3dd63a17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04d12a097ea247cbb0242c8d346a7117'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:28.430009 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-dd2c3174-f474-412d-9502-d8bd3dd63a17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04d12a097ea247cbb0242c8d346a7117'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:28.430758 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-dd2c3174-f474-412d-9502-d8bd3dd63a17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04d12a097ea247cbb0242c8d346a7117'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:28.443084 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-dd2c3174-f474-412d-9502-d8bd3dd63a17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04d12a097ea247cbb0242c8d346a7117'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:28.454807 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1449/2886] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 11:06:28 2022] DELETE /identity/v3/users/04d12a097ea247cbb0242c8d346a7117 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 11:06:28.719308 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b264ba52-c5d5-4ee6-88a2-ba830e763932 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:28.720803 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-b264ba52-c5d5-4ee6-88a2-ba830e763932 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:06:28.735732 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b264ba52-c5d5-4ee6-88a2-ba830e763932 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:28.793868 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a4a52e-45c0-48ec-9cf0-09ebacff9708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:28.794549 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a4a52e-45c0-48ec-9cf0-09ebacff9708 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:28.794775 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a4a52e-45c0-48ec-9cf0-09ebacff9708 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:28.795048 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a4a52e-45c0-48ec-9cf0-09ebacff9708 admin admin] PATH_INFO: `/v3/projects/83349722398d4170821d3c860ac8cf18` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:28.796192 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a4a52e-45c0-48ec-9cf0-09ebacff9708 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83349722398d4170821d3c860ac8cf18)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:28.798092 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a4a52e-45c0-48ec-9cf0-09ebacff9708 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:28.843472 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-28a4a52e-45c0-48ec-9cf0-09ebacff9708 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83349722398d4170821d3c860ac8cf18', 'request_id': 'req-28a4a52e-45c0-48ec-9cf0-09ebacff9708'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:28.850312 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-28a4a52e-45c0-48ec-9cf0-09ebacff9708 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83349722398d4170821d3c860ac8cf18', 'request_id': 'req-28a4a52e-45c0-48ec-9cf0-09ebacff9708'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:28.857757 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-28a4a52e-45c0-48ec-9cf0-09ebacff9708 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83349722398d4170821d3c860ac8cf18', 'request_id': 'req-28a4a52e-45c0-48ec-9cf0-09ebacff9708'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:28.869619 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-28a4a52e-45c0-48ec-9cf0-09ebacff9708 admin admin] The token cache is being invalidate because project 83349722398d4170821d3c860ac8cf18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90653) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 11:06:28.869913 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-28a4a52e-45c0-48ec-9cf0-09ebacff9708 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:28.870637 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-28a4a52e-45c0-48ec-9cf0-09ebacff9708 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:28.872212 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1438/2887] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 11:06:28 2022] DELETE /identity/v3/projects/83349722398d4170821d3c860ac8cf18 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 11:06:28.876301 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2c3174-f474-412d-9502-d8bd3dd63a17 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:28.877750 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-dd2c3174-f474-412d-9502-d8bd3dd63a17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:06:28.890741 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2c3174-f474-412d-9502-d8bd3dd63a17 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:28.945833 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58095c07-ab01-4025-8144-15c56ac64170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:28.946457 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58095c07-ab01-4025-8144-15c56ac64170 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:28.946736 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58095c07-ab01-4025-8144-15c56ac64170 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:28.947004 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58095c07-ab01-4025-8144-15c56ac64170 admin admin] PATH_INFO: `/v3/users/d7a88ce9806e4e2cb6a87be4c6712d96` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:28.948113 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58095c07-ab01-4025-8144-15c56ac64170 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7a88ce9806e4e2cb6a87be4c6712d96)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:28.950009 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58095c07-ab01-4025-8144-15c56ac64170 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:28.985765 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 11:06:29.000319 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-58095c07-ab01-4025-8144-15c56ac64170 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7a88ce9806e4e2cb6a87be4c6712d96'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:29.007510 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-58095c07-ab01-4025-8144-15c56ac64170 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7a88ce9806e4e2cb6a87be4c6712d96'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:29.008198 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-58095c07-ab01-4025-8144-15c56ac64170 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7a88ce9806e4e2cb6a87be4c6712d96'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:29.008804 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-58095c07-ab01-4025-8144-15c56ac64170 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7a88ce9806e4e2cb6a87be4c6712d96'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:29.020517 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-58095c07-ab01-4025-8144-15c56ac64170 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7a88ce9806e4e2cb6a87be4c6712d96'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:29.032285 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1450/2888] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 11:06:28 2022] DELETE /identity/v3/users/d7a88ce9806e4e2cb6a87be4c6712d96 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 11:06:29.307200 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a4a52e-45c0-48ec-9cf0-09ebacff9708 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:29.308589 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-28a4a52e-45c0-48ec-9cf0-09ebacff9708 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:06:29.322400 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a4a52e-45c0-48ec-9cf0-09ebacff9708 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:29.375704 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 11:06:29.384218 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62d6d81-4d62-4648-91dc-ab2dc5383a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:29.385139 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d6d81-4d62-4648-91dc-ab2dc5383a25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:29.385556 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d6d81-4d62-4648-91dc-ab2dc5383a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:29.385826 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d6d81-4d62-4648-91dc-ab2dc5383a25 admin admin] PATH_INFO: `/v3/projects/3639b1facaa8495897123ced2e4e101e` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:29.387060 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62d6d81-4d62-4648-91dc-ab2dc5383a25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3639b1facaa8495897123ced2e4e101e)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:29.389307 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62d6d81-4d62-4648-91dc-ab2dc5383a25 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:29.436063 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-b62d6d81-4d62-4648-91dc-ab2dc5383a25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3639b1facaa8495897123ced2e4e101e', 'request_id': 'req-b62d6d81-4d62-4648-91dc-ab2dc5383a25'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:29.442824 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-b62d6d81-4d62-4648-91dc-ab2dc5383a25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3639b1facaa8495897123ced2e4e101e', 'request_id': 'req-b62d6d81-4d62-4648-91dc-ab2dc5383a25'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:29.449797 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-b62d6d81-4d62-4648-91dc-ab2dc5383a25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3639b1facaa8495897123ced2e4e101e', 'request_id': 'req-b62d6d81-4d62-4648-91dc-ab2dc5383a25'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:29.460725 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-b62d6d81-4d62-4648-91dc-ab2dc5383a25 admin admin] The token cache is being invalidate because project 3639b1facaa8495897123ced2e4e101e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90653) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 11:06:29.461013 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-b62d6d81-4d62-4648-91dc-ab2dc5383a25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:29.461659 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-b62d6d81-4d62-4648-91dc-ab2dc5383a25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:29.463131 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1439/2889] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 11:06:29 2022] DELETE /identity/v3/projects/3639b1facaa8495897123ced2e4e101e => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 11:06:29.469783 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecd6e3a-7e91-4ac2-9c49-330a1a518f88 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:29.470075 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecd6e3a-7e91-4ac2-9c49-330a1a518f88 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:29.470350 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecd6e3a-7e91-4ac2-9c49-330a1a518f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:29.490075 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-fecd6e3a-7e91-4ac2-9c49-330a1a518f88 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 11:06:29.491269 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-fecd6e3a-7e91-4ac2-9c49-330a1a518f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:06:29.549161 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1451/2890] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 11:06:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 11:06:29.557611 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a28b64e-1eaa-4384-b8b3-9c54215a0f33 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:29.557921 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a28b64e-1eaa-4384-b8b3-9c54215a0f33 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:29.558223 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a28b64e-1eaa-4384-b8b3-9c54215a0f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:29.579002 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-4a28b64e-1eaa-4384-b8b3-9c54215a0f33 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 11:06:29.580226 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-4a28b64e-1eaa-4384-b8b3-9c54215a0f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:06:29.588527 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1440/2891] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 11:06:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 11:06:29.592616 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecd6e3a-7e91-4ac2-9c49-330a1a518f88 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:29.602162 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecd6e3a-7e91-4ac2-9c49-330a1a518f88 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:29.606642 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6fb80e-7777-4430-ac24-3ec6e928f8b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:29.607232 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6fb80e-7777-4430-ac24-3ec6e928f8b7 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:29.607541 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6fb80e-7777-4430-ac24-3ec6e928f8b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:29.607811 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6fb80e-7777-4430-ac24-3ec6e928f8b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:29.608441 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6fb80e-7777-4430-ac24-3ec6e928f8b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=90654) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 11:06:29.608738 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6fb80e-7777-4430-ac24-3ec6e928f8b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:29.610313 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6fb80e-7777-4430-ac24-3ec6e928f8b7 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:29.622208 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1452/2892] 10.222.0.15 () {58 vars in 1153 bytes} [Sun Sep 11 11:06:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 11:06:29.626486 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a28b64e-1eaa-4384-b8b3-9c54215a0f33 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:29.636450 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a28b64e-1eaa-4384-b8b3-9c54215a0f33 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:29.640941 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c524a7-ed22-4316-9b74-24e56b8c4076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:29.641522 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c524a7-ed22-4316-9b74-24e56b8c4076 admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:29.641801 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c524a7-ed22-4316-9b74-24e56b8c4076 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:29.642070 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c524a7-ed22-4316-9b74-24e56b8c4076 admin admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:29.642705 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c524a7-ed22-4316-9b74-24e56b8c4076 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:29.644717 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c524a7-ed22-4316-9b74-24e56b8c4076 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:29.671314 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1441/2893] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 11:06:29 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 11:06:29.675932 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6fb80e-7777-4430-ac24-3ec6e928f8b7 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:29.685787 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6fb80e-7777-4430-ac24-3ec6e928f8b7 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:29.740580 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3defbaeb-0df4-48ee-b8a5-4da2e9ebed62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:29.741173 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3defbaeb-0df4-48ee-b8a5-4da2e9ebed62 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:29.741455 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3defbaeb-0df4-48ee-b8a5-4da2e9ebed62 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:29.741728 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3defbaeb-0df4-48ee-b8a5-4da2e9ebed62 admin admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:29.742348 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3defbaeb-0df4-48ee-b8a5-4da2e9ebed62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:29.744304 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3defbaeb-0df4-48ee-b8a5-4da2e9ebed62 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:29.765272 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1453/2894] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 11:06:29 2022] POST /identity/v3/users => generated 293 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 11:06:29.769884 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c524a7-ed22-4316-9b74-24e56b8c4076 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:29.780065 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c524a7-ed22-4316-9b74-24e56b8c4076 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:29.784630 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372f9370-c1cd-49d3-b104-e747b2f41e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:29.785402 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372f9370-c1cd-49d3-b104-e747b2f41e2a admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:29.785681 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372f9370-c1cd-49d3-b104-e747b2f41e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:29.785952 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372f9370-c1cd-49d3-b104-e747b2f41e2a admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:29.786583 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372f9370-c1cd-49d3-b104-e747b2f41e2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:29.788227 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372f9370-c1cd-49d3-b104-e747b2f41e2a admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:29.797963 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1442/2895] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 11:06:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 11:06:29.802540 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3defbaeb-0df4-48ee-b8a5-4da2e9ebed62 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:29.812402 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3defbaeb-0df4-48ee-b8a5-4da2e9ebed62 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:29.817596 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc8cc9f-8105-4681-ae1b-135392d8ba4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:29.818195 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8cc9f-8105-4681-ae1b-135392d8ba4e admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:29.818476 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8cc9f-8105-4681-ae1b-135392d8ba4e admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:29.818775 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8cc9f-8105-4681-ae1b-135392d8ba4e admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:29.819408 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc8cc9f-8105-4681-ae1b-135392d8ba4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:29.821062 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc8cc9f-8105-4681-ae1b-135392d8ba4e admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:29.831664 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1454/2896] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 11:06:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 11:06:29.835604 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372f9370-c1cd-49d3-b104-e747b2f41e2a admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:29.845318 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372f9370-c1cd-49d3-b104-e747b2f41e2a admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:29.849958 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872d9456-4713-4031-9143-50bc7330c12c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:29.850595 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d9456-4713-4031-9143-50bc7330c12c admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:29.850873 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d9456-4713-4031-9143-50bc7330c12c admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:29.851141 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d9456-4713-4031-9143-50bc7330c12c admin admin] PATH_INFO: `/v3/projects/43c0148196d346a2961900eff2888f6c/users/457d3db9c81649538e6de2ca4c453a1f/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:29.867224 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872d9456-4713-4031-9143-50bc7330c12c admin admin] RBAC: Authorizing `identity:create_grant(project_id=43c0148196d346a2961900eff2888f6c, user_id=457d3db9c81649538e6de2ca4c453a1f, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:29.869953 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872d9456-4713-4031-9143-50bc7330c12c admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:29.882858 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1443/2897] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 11:06:29 2022] PUT /identity/v3/projects/43c0148196d346a2961900eff2888f6c/users/457d3db9c81649538e6de2ca4c453a1f/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 11:06:29.895879 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bcbc6c-8115-4f0b-96c6-afdf7e75b545 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:29.896388 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bcbc6c-8115-4f0b-96c6-afdf7e75b545 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:29.896615 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bcbc6c-8115-4f0b-96c6-afdf7e75b545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:29.924725 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-06bcbc6c-8115-4f0b-96c6-afdf7e75b545 None None] MFA Rules not processed for user `457d3db9c81649538e6de2ca4c453a1f`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 11:06:29.926128 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-06bcbc6c-8115-4f0b-96c6-afdf7e75b545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:06:29.929975 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 11:06:29.991142 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1455/2898] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 11:06:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 11:06:29.995671 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872d9456-4713-4031-9143-50bc7330c12c admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:30.056403 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872d9456-4713-4031-9143-50bc7330c12c admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:30.061000 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da20e405-2ba7-414b-b4f6-3fdbeded4286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:30.061600 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da20e405-2ba7-414b-b4f6-3fdbeded4286 admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:30.061869 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da20e405-2ba7-414b-b4f6-3fdbeded4286 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:30.062138 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da20e405-2ba7-414b-b4f6-3fdbeded4286 admin admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:30.062758 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da20e405-2ba7-414b-b4f6-3fdbeded4286 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:30.064706 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da20e405-2ba7-414b-b4f6-3fdbeded4286 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:30.091245 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1444/2899] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 11:06:29 2022] POST /identity/v3/projects => generated 383 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 11:06:30.095197 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bcbc6c-8115-4f0b-96c6-afdf7e75b545 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:30.106128 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bcbc6c-8115-4f0b-96c6-afdf7e75b545 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:30.162163 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab9668c-27d5-4307-8425-df1e0a0c6952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:30.162703 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab9668c-27d5-4307-8425-df1e0a0c6952 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:30.162986 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab9668c-27d5-4307-8425-df1e0a0c6952 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:30.163261 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab9668c-27d5-4307-8425-df1e0a0c6952 admin admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:30.163931 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab9668c-27d5-4307-8425-df1e0a0c6952 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:30.165838 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab9668c-27d5-4307-8425-df1e0a0c6952 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:30.186188 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1456/2900] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 11:06:30 2022] POST /identity/v3/users => generated 292 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 11:06:30.190506 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da20e405-2ba7-414b-b4f6-3fdbeded4286 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:30.201892 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da20e405-2ba7-414b-b4f6-3fdbeded4286 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:30.207999 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b0e771-ada5-43fc-a540-2edebc89fe69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:30.208614 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b0e771-ada5-43fc-a540-2edebc89fe69 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:30.208897 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b0e771-ada5-43fc-a540-2edebc89fe69 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:30.209173 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b0e771-ada5-43fc-a540-2edebc89fe69 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:30.209840 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b0e771-ada5-43fc-a540-2edebc89fe69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:30.211452 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b0e771-ada5-43fc-a540-2edebc89fe69 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:30.212767 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 11:06:30.223808 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1445/2901] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 11:06:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 11:06:30.227589 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab9668c-27d5-4307-8425-df1e0a0c6952 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:30.237755 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab9668c-27d5-4307-8425-df1e0a0c6952 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:30.242658 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fed606-7374-4811-9c69-49bf9f4c152f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:30.243254 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fed606-7374-4811-9c69-49bf9f4c152f admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:30.243574 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fed606-7374-4811-9c69-49bf9f4c152f admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:30.243970 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fed606-7374-4811-9c69-49bf9f4c152f admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:30.244617 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fed606-7374-4811-9c69-49bf9f4c152f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:30.246244 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fed606-7374-4811-9c69-49bf9f4c152f admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:30.256721 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1457/2902] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 11:06:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 11:06:30.260875 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b0e771-ada5-43fc-a540-2edebc89fe69 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:30.271716 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b0e771-ada5-43fc-a540-2edebc89fe69 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:30.276343 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971195cc-a7a3-406b-8681-40a22786da71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:30.276978 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971195cc-a7a3-406b-8681-40a22786da71 admin admin] REQUEST_METHOD: `PUT` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:30.277253 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971195cc-a7a3-406b-8681-40a22786da71 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:30.277522 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971195cc-a7a3-406b-8681-40a22786da71 admin admin] PATH_INFO: `/v3/projects/e22ff52bbf6f420ea924568906aa2aee/users/6c650d709294499da6ccc60d0f211d20/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:30.293353 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971195cc-a7a3-406b-8681-40a22786da71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e22ff52bbf6f420ea924568906aa2aee, user_id=6c650d709294499da6ccc60d0f211d20, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:30.296036 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971195cc-a7a3-406b-8681-40a22786da71 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:30.309633 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1446/2903] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 11:06:30 2022] PUT /identity/v3/projects/e22ff52bbf6f420ea924568906aa2aee/users/6c650d709294499da6ccc60d0f211d20/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 11:06:30.320733 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232a04b2-ef9a-4e7f-900f-4365e9a1d196 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:30.321124 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232a04b2-ef9a-4e7f-900f-4365e9a1d196 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:30.321408 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232a04b2-ef9a-4e7f-900f-4365e9a1d196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:30.349645 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-232a04b2-ef9a-4e7f-900f-4365e9a1d196 None None] MFA Rules not processed for user `6c650d709294499da6ccc60d0f211d20`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 11:06:30.350860 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-232a04b2-ef9a-4e7f-900f-4365e9a1d196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:06:30.414898 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1458/2904] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 11:06:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 11:06:30.419574 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971195cc-a7a3-406b-8681-40a22786da71 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:30.480634 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971195cc-a7a3-406b-8681-40a22786da71 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:30.485202 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9418c1f4-8171-47c4-8eb1-c6bd0912e41e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:30.485799 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9418c1f4-8171-47c4-8eb1-c6bd0912e41e admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:30.486078 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9418c1f4-8171-47c4-8eb1-c6bd0912e41e admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:30.486348 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9418c1f4-8171-47c4-8eb1-c6bd0912e41e admin admin] PATH_INFO: `/v3/projects` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:30.486965 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9418c1f4-8171-47c4-8eb1-c6bd0912e41e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:30.489779 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9418c1f4-8171-47c4-8eb1-c6bd0912e41e admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:30.515877 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1447/2905] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 11:06:30 2022] POST /identity/v3/projects => generated 383 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 11:06:30.520388 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232a04b2-ef9a-4e7f-900f-4365e9a1d196 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:30.530760 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232a04b2-ef9a-4e7f-900f-4365e9a1d196 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:30.539770 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 11:06:30.592607 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49764e21-9229-4324-b4b6-a36b4fc1a073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:30.593221 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49764e21-9229-4324-b4b6-a36b4fc1a073 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:30.593506 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49764e21-9229-4324-b4b6-a36b4fc1a073 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:30.593776 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49764e21-9229-4324-b4b6-a36b4fc1a073 admin admin] PATH_INFO: `/v3/users` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:30.594393 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49764e21-9229-4324-b4b6-a36b4fc1a073 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:30.596326 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49764e21-9229-4324-b4b6-a36b4fc1a073 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:30.617192 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1459/2906] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 11:06:30 2022] POST /identity/v3/users => generated 298 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 11:06:30.621063 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9418c1f4-8171-47c4-8eb1-c6bd0912e41e admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:30.631766 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9418c1f4-8171-47c4-8eb1-c6bd0912e41e admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:30.636296 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d435b4-87c5-4f0b-8152-7c175d0c1439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:30.636878 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d435b4-87c5-4f0b-8152-7c175d0c1439 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:30.637155 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d435b4-87c5-4f0b-8152-7c175d0c1439 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:30.637425 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d435b4-87c5-4f0b-8152-7c175d0c1439 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:30.638059 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d435b4-87c5-4f0b-8152-7c175d0c1439 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:30.639657 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d435b4-87c5-4f0b-8152-7c175d0c1439 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:30.649947 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1448/2907] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 11:06:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 11:06:30.653788 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49764e21-9229-4324-b4b6-a36b4fc1a073 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:30.663579 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49764e21-9229-4324-b4b6-a36b4fc1a073 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:30.668143 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a240fc-79ec-4298-b2b2-fb6639b90b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:30.668721 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a240fc-79ec-4298-b2b2-fb6639b90b25 admin admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:30.668999 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a240fc-79ec-4298-b2b2-fb6639b90b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:30.669269 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a240fc-79ec-4298-b2b2-fb6639b90b25 admin admin] PATH_INFO: `/v3/projects/8b70a862cf9a4546b51634b7e846ff8c/users/25c6e364322346f885f8af16913d91ff/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:30.684634 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a240fc-79ec-4298-b2b2-fb6639b90b25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b70a862cf9a4546b51634b7e846ff8c, user_id=25c6e364322346f885f8af16913d91ff, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:30.687332 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a240fc-79ec-4298-b2b2-fb6639b90b25 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:30.700818 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1460/2908] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 11:06:30 2022] PUT /identity/v3/projects/8b70a862cf9a4546b51634b7e846ff8c/users/25c6e364322346f885f8af16913d91ff/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 11:06:30.704749 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d435b4-87c5-4f0b-8152-7c175d0c1439 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:30.714719 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d435b4-87c5-4f0b-8152-7c175d0c1439 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:30.770702 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92729ef8-9faa-44bc-be66-74d8a6ed27ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:30.771304 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92729ef8-9faa-44bc-be66-74d8a6ed27ab admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:30.771615 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92729ef8-9faa-44bc-be66-74d8a6ed27ab admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:30.771887 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92729ef8-9faa-44bc-be66-74d8a6ed27ab admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:30.772529 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92729ef8-9faa-44bc-be66-74d8a6ed27ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:30.774126 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92729ef8-9faa-44bc-be66-74d8a6ed27ab admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:30.783751 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1449/2909] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 11:06:30 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 11:06:30.788760 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a240fc-79ec-4298-b2b2-fb6639b90b25 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:30.798668 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a240fc-79ec-4298-b2b2-fb6639b90b25 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:30.804252 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65393684-e4ad-41f9-ad70-253e63e5327a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:30.804892 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65393684-e4ad-41f9-ad70-253e63e5327a admin admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:30.805169 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65393684-e4ad-41f9-ad70-253e63e5327a admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:30.805441 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65393684-e4ad-41f9-ad70-253e63e5327a admin admin] PATH_INFO: `/v3/projects/8b70a862cf9a4546b51634b7e846ff8c/users/25c6e364322346f885f8af16913d91ff/roles/2cb7d73f0ea8414296367573f6902340` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:30.807307 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65393684-e4ad-41f9-ad70-253e63e5327a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b70a862cf9a4546b51634b7e846ff8c, user_id=25c6e364322346f885f8af16913d91ff, role_id=2cb7d73f0ea8414296367573f6902340)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:30.809943 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65393684-e4ad-41f9-ad70-253e63e5327a admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:30.820107 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1461/2910] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 11:06:30 2022] PUT /identity/v3/projects/8b70a862cf9a4546b51634b7e846ff8c/users/25c6e364322346f885f8af16913d91ff/roles/2cb7d73f0ea8414296367573f6902340 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 11:06:30.829885 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6860fe-0c17-4bd7-88b9-3ce65ea9e658 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:30.830352 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6860fe-0c17-4bd7-88b9-3ce65ea9e658 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:30.830454 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6860fe-0c17-4bd7-88b9-3ce65ea9e658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:30.858216 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-fd6860fe-0c17-4bd7-88b9-3ce65ea9e658 None None] MFA Rules not processed for user `25c6e364322346f885f8af16913d91ff`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 11:06:30.859544 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-fd6860fe-0c17-4bd7-88b9-3ce65ea9e658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:06:30.890652 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 11:06:30.931638 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1450/2911] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 11:06:30 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 11:06:30.944374 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65393684-e4ad-41f9-ad70-253e63e5327a admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:30.946078 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-65393684-e4ad-41f9-ad70-253e63e5327a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:06:31.006712 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65393684-e4ad-41f9-ad70-253e63e5327a admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:31.010372 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820e5cd5-7185-473f-b3d3-34b2dfaf8c78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:31.010973 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820e5cd5-7185-473f-b3d3-34b2dfaf8c78 service cinder] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:31.011258 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820e5cd5-7185-473f-b3d3-34b2dfaf8c78 service cinder] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:31.011628 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820e5cd5-7185-473f-b3d3-34b2dfaf8c78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:31.067033 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820e5cd5-7185-473f-b3d3-34b2dfaf8c78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:31.068501 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820e5cd5-7185-473f-b3d3-34b2dfaf8c78 service cinder] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:31.077602 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1462/2912] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 11:06:30 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 11:06:32.582671 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6860fe-0c17-4bd7-88b9-3ce65ea9e658 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:32.584084 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-fd6860fe-0c17-4bd7-88b9-3ce65ea9e658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:06:32.595944 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6860fe-0c17-4bd7-88b9-3ce65ea9e658 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:32.600284 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b70523c-18aa-4738-9e12-d0752ca01d22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:32.600876 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b70523c-18aa-4738-9e12-d0752ca01d22 service cinder] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:32.601155 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b70523c-18aa-4738-9e12-d0752ca01d22 service cinder] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:32.601426 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b70523c-18aa-4738-9e12-d0752ca01d22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:32.656877 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b70523c-18aa-4738-9e12-d0752ca01d22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:32.658289 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b70523c-18aa-4738-9e12-d0752ca01d22 service cinder] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:32.667805 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1451/2913] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 11:06:32 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 11:06:33.193459 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820e5cd5-7185-473f-b3d3-34b2dfaf8c78 service cinder] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:33.201964 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820e5cd5-7185-473f-b3d3-34b2dfaf8c78 service cinder] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:33.205561 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd70bb4-096c-4fa5-98ac-3ad62658cd59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:33.206159 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd70bb4-096c-4fa5-98ac-3ad62658cd59 service cinder] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:33.206437 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd70bb4-096c-4fa5-98ac-3ad62658cd59 service cinder] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:33.206708 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd70bb4-096c-4fa5-98ac-3ad62658cd59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:33.215999 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd70bb4-096c-4fa5-98ac-3ad62658cd59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:33.217415 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd70bb4-096c-4fa5-98ac-3ad62658cd59 service cinder] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:33.226823 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1463/2914] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 11:06:33 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 11:06:37.103230 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b70523c-18aa-4738-9e12-d0752ca01d22 service cinder] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:37.165681 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b70523c-18aa-4738-9e12-d0752ca01d22 service cinder] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:37.170055 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55952a3d-6f0c-4ab2-81d2-6fb4ac49bf3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:37.170662 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55952a3d-6f0c-4ab2-81d2-6fb4ac49bf3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:37.170938 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55952a3d-6f0c-4ab2-81d2-6fb4ac49bf3e admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:37.171204 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55952a3d-6f0c-4ab2-81d2-6fb4ac49bf3e admin admin] PATH_INFO: `/v3/users/457d3db9c81649538e6de2ca4c453a1f` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:37.172318 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55952a3d-6f0c-4ab2-81d2-6fb4ac49bf3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=457d3db9c81649538e6de2ca4c453a1f)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:37.174216 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55952a3d-6f0c-4ab2-81d2-6fb4ac49bf3e admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:37.218268 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-55952a3d-6f0c-4ab2-81d2-6fb4ac49bf3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '457d3db9c81649538e6de2ca4c453a1f'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:37.226146 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-55952a3d-6f0c-4ab2-81d2-6fb4ac49bf3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '457d3db9c81649538e6de2ca4c453a1f'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:37.227044 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-55952a3d-6f0c-4ab2-81d2-6fb4ac49bf3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '457d3db9c81649538e6de2ca4c453a1f'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:37.227772 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-55952a3d-6f0c-4ab2-81d2-6fb4ac49bf3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '457d3db9c81649538e6de2ca4c453a1f'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:37.234843 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-55952a3d-6f0c-4ab2-81d2-6fb4ac49bf3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '457d3db9c81649538e6de2ca4c453a1f'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:37.239605 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 11:06:37.254542 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1452/2915] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 11:06:37 2022] DELETE /identity/v3/users/457d3db9c81649538e6de2ca4c453a1f => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 11:06:37.259013 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6612bfec-4819-4ded-b4eb-9eb518423d4e None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:37.259536 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6612bfec-4819-4ded-b4eb-9eb518423d4e None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:37.259643 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6612bfec-4819-4ded-b4eb-9eb518423d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:37.273070 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 11:06:37.279173 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-6612bfec-4819-4ded-b4eb-9eb518423d4e None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 11:06:37.281017 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-6612bfec-4819-4ded-b4eb-9eb518423d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:06:37.341283 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1464/2916] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 11:06:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 11:06:37.351126 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55952a3d-6f0c-4ab2-81d2-6fb4ac49bf3e admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:37.352605 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-55952a3d-6f0c-4ab2-81d2-6fb4ac49bf3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:06:37.409223 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55952a3d-6f0c-4ab2-81d2-6fb4ac49bf3e admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:37.413532 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423ead8c-b88a-47e1-8b04-e0a73964a0b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:37.414131 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423ead8c-b88a-47e1-8b04-e0a73964a0b7 service neutron] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:37.414407 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423ead8c-b88a-47e1-8b04-e0a73964a0b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:37.414676 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423ead8c-b88a-47e1-8b04-e0a73964a0b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:37.423462 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423ead8c-b88a-47e1-8b04-e0a73964a0b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:37.424870 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423ead8c-b88a-47e1-8b04-e0a73964a0b7 service neutron] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:37.434795 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1453/2917] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 11:06:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 11:06:37.699224 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6612bfec-4819-4ded-b4eb-9eb518423d4e None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:37.700729 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-6612bfec-4819-4ded-b4eb-9eb518423d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:06:37.712759 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6612bfec-4819-4ded-b4eb-9eb518423d4e None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:37.717700 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa1bf05-c411-40f5-9782-6f86e72b97da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:37.718335 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa1bf05-c411-40f5-9782-6f86e72b97da admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:37.718614 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa1bf05-c411-40f5-9782-6f86e72b97da admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:37.718896 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa1bf05-c411-40f5-9782-6f86e72b97da admin admin] PATH_INFO: `/v3/projects/43c0148196d346a2961900eff2888f6c` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:37.720030 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa1bf05-c411-40f5-9782-6f86e72b97da admin admin] RBAC: Authorizing `identity:delete_project(project_id=43c0148196d346a2961900eff2888f6c)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:37.721938 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa1bf05-c411-40f5-9782-6f86e72b97da admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:37.769069 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-4aa1bf05-c411-40f5-9782-6f86e72b97da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43c0148196d346a2961900eff2888f6c', 'request_id': 'req-4aa1bf05-c411-40f5-9782-6f86e72b97da'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:37.775385 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-4aa1bf05-c411-40f5-9782-6f86e72b97da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43c0148196d346a2961900eff2888f6c', 'request_id': 'req-4aa1bf05-c411-40f5-9782-6f86e72b97da'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:37.781023 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-4aa1bf05-c411-40f5-9782-6f86e72b97da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43c0148196d346a2961900eff2888f6c', 'request_id': 'req-4aa1bf05-c411-40f5-9782-6f86e72b97da'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:37.794274 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-4aa1bf05-c411-40f5-9782-6f86e72b97da admin admin] The token cache is being invalidate because project 43c0148196d346a2961900eff2888f6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 11:06:37.794563 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-4aa1bf05-c411-40f5-9782-6f86e72b97da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:37.795181 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-4aa1bf05-c411-40f5-9782-6f86e72b97da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:37.796659 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1465/2918] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 11:06:37 2022] DELETE /identity/v3/projects/43c0148196d346a2961900eff2888f6c => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 11:06:37.800671 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423ead8c-b88a-47e1-8b04-e0a73964a0b7 service neutron] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:37.810979 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423ead8c-b88a-47e1-8b04-e0a73964a0b7 service neutron] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:37.866907 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d24794-8daf-431c-b43b-5523086ed4be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:37.867490 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d24794-8daf-431c-b43b-5523086ed4be admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:37.867772 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d24794-8daf-431c-b43b-5523086ed4be admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:37.868044 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d24794-8daf-431c-b43b-5523086ed4be admin admin] PATH_INFO: `/v3/users/6c650d709294499da6ccc60d0f211d20` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:37.869161 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d24794-8daf-431c-b43b-5523086ed4be admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c650d709294499da6ccc60d0f211d20)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:37.871076 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d24794-8daf-431c-b43b-5523086ed4be admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:37.915549 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-35d24794-8daf-431c-b43b-5523086ed4be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c650d709294499da6ccc60d0f211d20'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:37.922773 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-35d24794-8daf-431c-b43b-5523086ed4be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c650d709294499da6ccc60d0f211d20'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:37.923400 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-35d24794-8daf-431c-b43b-5523086ed4be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c650d709294499da6ccc60d0f211d20'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:37.924033 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-35d24794-8daf-431c-b43b-5523086ed4be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c650d709294499da6ccc60d0f211d20'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:37.931791 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-35d24794-8daf-431c-b43b-5523086ed4be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c650d709294499da6ccc60d0f211d20'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:37.949012 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1454/2919] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 11:06:37 2022] DELETE /identity/v3/users/6c650d709294499da6ccc60d0f211d20 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 11:06:38.215133 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa1bf05-c411-40f5-9782-6f86e72b97da admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:38.216740 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-4aa1bf05-c411-40f5-9782-6f86e72b97da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:06:38.231066 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa1bf05-c411-40f5-9782-6f86e72b97da admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:38.289915 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7150169-a1a5-4e23-afdc-0ba09642c840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:38.290618 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7150169-a1a5-4e23-afdc-0ba09642c840 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:38.290811 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7150169-a1a5-4e23-afdc-0ba09642c840 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:38.291085 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7150169-a1a5-4e23-afdc-0ba09642c840 admin admin] PATH_INFO: `/v3/projects/e22ff52bbf6f420ea924568906aa2aee` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:38.292217 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7150169-a1a5-4e23-afdc-0ba09642c840 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e22ff52bbf6f420ea924568906aa2aee)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:38.294105 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7150169-a1a5-4e23-afdc-0ba09642c840 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:38.328523 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 11:06:38.342010 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-d7150169-a1a5-4e23-afdc-0ba09642c840 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e22ff52bbf6f420ea924568906aa2aee', 'request_id': 'req-d7150169-a1a5-4e23-afdc-0ba09642c840'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:38.348645 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-d7150169-a1a5-4e23-afdc-0ba09642c840 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e22ff52bbf6f420ea924568906aa2aee', 'request_id': 'req-d7150169-a1a5-4e23-afdc-0ba09642c840'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:38.354219 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-d7150169-a1a5-4e23-afdc-0ba09642c840 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e22ff52bbf6f420ea924568906aa2aee', 'request_id': 'req-d7150169-a1a5-4e23-afdc-0ba09642c840'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:38.367757 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-d7150169-a1a5-4e23-afdc-0ba09642c840 admin admin] The token cache is being invalidate because project e22ff52bbf6f420ea924568906aa2aee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 11:06:38.368044 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-d7150169-a1a5-4e23-afdc-0ba09642c840 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:38.368672 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-d7150169-a1a5-4e23-afdc-0ba09642c840 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:38.370165 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1466/2920] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 11:06:38 2022] DELETE /identity/v3/projects/e22ff52bbf6f420ea924568906aa2aee => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 11:06:38.374602 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d24794-8daf-431c-b43b-5523086ed4be admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:38.376056 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-35d24794-8daf-431c-b43b-5523086ed4be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:06:38.389146 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d24794-8daf-431c-b43b-5523086ed4be admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:38.392691 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 11:06:38.447711 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152ba5aa-e306-4880-87b2-ec5101a0fbb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:38.448356 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152ba5aa-e306-4880-87b2-ec5101a0fbb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:38.448618 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152ba5aa-e306-4880-87b2-ec5101a0fbb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:38.448893 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152ba5aa-e306-4880-87b2-ec5101a0fbb8 admin admin] PATH_INFO: `/v3/users/25c6e364322346f885f8af16913d91ff` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:38.450010 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152ba5aa-e306-4880-87b2-ec5101a0fbb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25c6e364322346f885f8af16913d91ff)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:38.451959 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152ba5aa-e306-4880-87b2-ec5101a0fbb8 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:38.498097 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-152ba5aa-e306-4880-87b2-ec5101a0fbb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25c6e364322346f885f8af16913d91ff'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:38.505468 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-152ba5aa-e306-4880-87b2-ec5101a0fbb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25c6e364322346f885f8af16913d91ff'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:38.506074 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-152ba5aa-e306-4880-87b2-ec5101a0fbb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25c6e364322346f885f8af16913d91ff'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:38.506669 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-152ba5aa-e306-4880-87b2-ec5101a0fbb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25c6e364322346f885f8af16913d91ff'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:38.513711 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-152ba5aa-e306-4880-87b2-ec5101a0fbb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25c6e364322346f885f8af16913d91ff'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:38.530813 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1455/2921] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 11:06:38 2022] DELETE /identity/v3/users/25c6e364322346f885f8af16913d91ff => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 11:06:38.798307 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7150169-a1a5-4e23-afdc-0ba09642c840 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:38.799747 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-d7150169-a1a5-4e23-afdc-0ba09642c840 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:06:38.812655 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7150169-a1a5-4e23-afdc-0ba09642c840 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:38.869427 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab781245-918d-498f-aac1-5bce2d829215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:38.870061 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab781245-918d-498f-aac1-5bce2d829215 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:38.870337 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab781245-918d-498f-aac1-5bce2d829215 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:38.870607 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab781245-918d-498f-aac1-5bce2d829215 admin admin] PATH_INFO: `/v3/projects/8b70a862cf9a4546b51634b7e846ff8c` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:38.871721 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab781245-918d-498f-aac1-5bce2d829215 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b70a862cf9a4546b51634b7e846ff8c)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:38.873647 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab781245-918d-498f-aac1-5bce2d829215 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:38.920414 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-ab781245-918d-498f-aac1-5bce2d829215 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b70a862cf9a4546b51634b7e846ff8c', 'request_id': 'req-ab781245-918d-498f-aac1-5bce2d829215'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:38.926504 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-ab781245-918d-498f-aac1-5bce2d829215 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b70a862cf9a4546b51634b7e846ff8c', 'request_id': 'req-ab781245-918d-498f-aac1-5bce2d829215'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:38.931928 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-ab781245-918d-498f-aac1-5bce2d829215 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b70a862cf9a4546b51634b7e846ff8c', 'request_id': 'req-ab781245-918d-498f-aac1-5bce2d829215'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:38.944425 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-ab781245-918d-498f-aac1-5bce2d829215 admin admin] The token cache is being invalidate because project 8b70a862cf9a4546b51634b7e846ff8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 11:06:38.944713 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-ab781245-918d-498f-aac1-5bce2d829215 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:38.945326 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-ab781245-918d-498f-aac1-5bce2d829215 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:38.946822 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1467/2922] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 11:06:38 2022] DELETE /identity/v3/projects/8b70a862cf9a4546b51634b7e846ff8c => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 11:06:38.952673 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60cfd2c-3c76-4e2d-a732-d33ec87d5a0d None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:38.952957 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60cfd2c-3c76-4e2d-a732-d33ec87d5a0d None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:38.953225 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60cfd2c-3c76-4e2d-a732-d33ec87d5a0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:38.973529 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-b60cfd2c-3c76-4e2d-a732-d33ec87d5a0d None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 11:06:38.974760 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-b60cfd2c-3c76-4e2d-a732-d33ec87d5a0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:06:39.035060 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1456/2923] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 11:06:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 11:06:39.043050 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e960bc62-0a29-4741-a014-323224c821ca None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:39.043340 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e960bc62-0a29-4741-a014-323224c821ca None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:39.043663 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e960bc62-0a29-4741-a014-323224c821ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:39.064309 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-e960bc62-0a29-4741-a014-323224c821ca None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 11:06:39.065501 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-e960bc62-0a29-4741-a014-323224c821ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:06:39.073823 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1468/2924] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 11:06:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 11:06:39.077918 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60cfd2c-3c76-4e2d-a732-d33ec87d5a0d None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:39.088587 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60cfd2c-3c76-4e2d-a732-d33ec87d5a0d None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:39.093495 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5526cd73-537f-430d-873f-8015dab57fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:39.094098 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5526cd73-537f-430d-873f-8015dab57fcd admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:39.094406 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5526cd73-537f-430d-873f-8015dab57fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:39.094677 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5526cd73-537f-430d-873f-8015dab57fcd admin admin] PATH_INFO: `/v3/domains` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:39.095319 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5526cd73-537f-430d-873f-8015dab57fcd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=90653) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 11:06:39.095645 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5526cd73-537f-430d-873f-8015dab57fcd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:39.097235 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5526cd73-537f-430d-873f-8015dab57fcd admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:39.108997 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1457/2925] 10.222.0.15 () {58 vars in 1153 bytes} [Sun Sep 11 11:06:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 11:06:39.113564 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e960bc62-0a29-4741-a014-323224c821ca None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:39.122741 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e960bc62-0a29-4741-a014-323224c821ca None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:39.127313 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b94c40-243d-4f4d-bfa3-8a1422773725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:39.127934 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b94c40-243d-4f4d-bfa3-8a1422773725 admin admin] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:39.128212 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b94c40-243d-4f4d-bfa3-8a1422773725 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:39.128484 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b94c40-243d-4f4d-bfa3-8a1422773725 admin admin] PATH_INFO: `/v3/projects` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:39.129099 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b94c40-243d-4f4d-bfa3-8a1422773725 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:39.131526 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b94c40-243d-4f4d-bfa3-8a1422773725 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:39.157971 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1469/2926] 10.222.0.15 () {60 vars in 1153 bytes} [Sun Sep 11 11:06:39 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 11:06:39.162430 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5526cd73-537f-430d-873f-8015dab57fcd admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:39.172377 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5526cd73-537f-430d-873f-8015dab57fcd admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:39.181634 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 11:06:39.231875 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26dda1b-850f-4bc5-9cda-340051fa57f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:39.232476 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26dda1b-850f-4bc5-9cda-340051fa57f5 admin admin] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:39.232750 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26dda1b-850f-4bc5-9cda-340051fa57f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:39.233018 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26dda1b-850f-4bc5-9cda-340051fa57f5 admin admin] PATH_INFO: `/v3/users` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:39.233631 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26dda1b-850f-4bc5-9cda-340051fa57f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:39.235547 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26dda1b-850f-4bc5-9cda-340051fa57f5 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:39.256802 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1458/2927] 10.222.0.15 () {60 vars in 1144 bytes} [Sun Sep 11 11:06:39 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 11:06:39.260887 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b94c40-243d-4f4d-bfa3-8a1422773725 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:39.271888 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b94c40-243d-4f4d-bfa3-8a1422773725 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:39.276443 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c9dfff-9c56-4317-bd90-e1946973dd67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:39.277042 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c9dfff-9c56-4317-bd90-e1946973dd67 admin admin] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:39.277324 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c9dfff-9c56-4317-bd90-e1946973dd67 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:39.277597 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c9dfff-9c56-4317-bd90-e1946973dd67 admin admin] PATH_INFO: `/v3/roles` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:39.278231 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c9dfff-9c56-4317-bd90-e1946973dd67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:39.280182 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c9dfff-9c56-4317-bd90-e1946973dd67 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:39.290773 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1470/2928] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 11:06:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 11:06:39.294727 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26dda1b-850f-4bc5-9cda-340051fa57f5 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:39.305378 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26dda1b-850f-4bc5-9cda-340051fa57f5 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:39.310162 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4826c413-68ef-4749-b56f-455f7a54edc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:39.310869 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826c413-68ef-4749-b56f-455f7a54edc0 admin admin] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:39.311172 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826c413-68ef-4749-b56f-455f7a54edc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:39.311446 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826c413-68ef-4749-b56f-455f7a54edc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:39.312101 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4826c413-68ef-4749-b56f-455f7a54edc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:39.313725 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4826c413-68ef-4749-b56f-455f7a54edc0 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:39.325199 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1459/2929] 10.222.0.15 () {58 vars in 1122 bytes} [Sun Sep 11 11:06:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 11:06:39.329225 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c9dfff-9c56-4317-bd90-e1946973dd67 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:39.339309 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c9dfff-9c56-4317-bd90-e1946973dd67 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:39.344061 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9130a867-21eb-44a0-8138-f8c59b1f6bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:39.344837 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9130a867-21eb-44a0-8138-f8c59b1f6bf7 admin admin] REQUEST_METHOD: `PUT` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:39.345330 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9130a867-21eb-44a0-8138-f8c59b1f6bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:39.345620 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9130a867-21eb-44a0-8138-f8c59b1f6bf7 admin admin] PATH_INFO: `/v3/projects/2e6c7071912645288d40f666ec2d4582/users/efaa6f24a9f941b0a3dd7d16dd46e56f/roles/f836487a66c74c05838133c9d0614bbf` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:39.354240 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 11:06:39.362217 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9130a867-21eb-44a0-8138-f8c59b1f6bf7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e6c7071912645288d40f666ec2d4582, user_id=efaa6f24a9f941b0a3dd7d16dd46e56f, role_id=f836487a66c74c05838133c9d0614bbf)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:39.366343 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9130a867-21eb-44a0-8138-f8c59b1f6bf7 admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:39.379217 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1471/2930] 10.222.0.15 () {60 vars in 1483 bytes} [Sun Sep 11 11:06:39 2022] PUT /identity/v3/projects/2e6c7071912645288d40f666ec2d4582/users/efaa6f24a9f941b0a3dd7d16dd46e56f/roles/f836487a66c74c05838133c9d0614bbf => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 11:06:39.390179 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cefaa8a-2f07-42a2-8d90-89b2cf3019e5 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:39.390665 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cefaa8a-2f07-42a2-8d90-89b2cf3019e5 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:39.390773 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cefaa8a-2f07-42a2-8d90-89b2cf3019e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:39.418177 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-8cefaa8a-2f07-42a2-8d90-89b2cf3019e5 None None] MFA Rules not processed for user `efaa6f24a9f941b0a3dd7d16dd46e56f`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 11:06:39.419409 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-8cefaa8a-2f07-42a2-8d90-89b2cf3019e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:06:39.484897 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1460/2931] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 11:06:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 11:06:39.496743 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9130a867-21eb-44a0-8138-f8c59b1f6bf7 admin admin] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:39.498417 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-9130a867-21eb-44a0-8138-f8c59b1f6bf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:06:39.553679 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9130a867-21eb-44a0-8138-f8c59b1f6bf7 admin admin] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:39.557233 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de00df19-6cae-4444-b456-16eac40e5d0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0bbb481abb8e43788e3412ac7d908ad3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:39.557831 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de00df19-6cae-4444-b456-16eac40e5d0f service cinder] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:39.558107 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de00df19-6cae-4444-b456-16eac40e5d0f service cinder] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:39.558376 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de00df19-6cae-4444-b456-16eac40e5d0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:39.566363 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de00df19-6cae-4444-b456-16eac40e5d0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:39.568425 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de00df19-6cae-4444-b456-16eac40e5d0f service cinder] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:39.576962 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1472/2932] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 11:06:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 11:06:40.935771 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d0c7de-63e0-49b1-a2cc-0ad5c1baa574 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:40.936371 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d0c7de-63e0-49b1-a2cc-0ad5c1baa574 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:40.936371 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d0c7de-63e0-49b1-a2cc-0ad5c1baa574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:40.955896 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-83d0c7de-63e0-49b1-a2cc-0ad5c1baa574 None None] MFA Rules not processed for user `efaa6f24a9f941b0a3dd7d16dd46e56f`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 11:06:40.957110 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-83d0c7de-63e0-49b1-a2cc-0ad5c1baa574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:06:40.964724 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1461/2933] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 11:06:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 11:06:41.454947 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de00df19-6cae-4444-b456-16eac40e5d0f service cinder] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:41.456423 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-de00df19-6cae-4444-b456-16eac40e5d0f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:06:41.515062 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de00df19-6cae-4444-b456-16eac40e5d0f service cinder] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:41.518681 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c47bf8-fa3b-404d-a0fd-0f5c2feb646c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:41.519288 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c47bf8-fa3b-404d-a0fd-0f5c2feb646c service neutron] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:41.519609 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c47bf8-fa3b-404d-a0fd-0f5c2feb646c service neutron] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:41.519886 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c47bf8-fa3b-404d-a0fd-0f5c2feb646c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:41.521663 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-96c47bf8-fa3b-404d-a0fd-0f5c2feb646c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:06:41.615770 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c47bf8-fa3b-404d-a0fd-0f5c2feb646c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:41.617275 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c47bf8-fa3b-404d-a0fd-0f5c2feb646c service neutron] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:41.640069 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1473/2934] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 11:06:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 11:06:50.950001 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=63,keystone:INSERT=3 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 11:06:51.636965 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=97,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 11:06:54.905866 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a882641-2fd5-4f8a-b81c-c6a29dcc7e0a None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:54.906342 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a882641-2fd5-4f8a-b81c-c6a29dcc7e0a None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:54.906461 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a882641-2fd5-4f8a-b81c-c6a29dcc7e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:54.935758 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-5a882641-2fd5-4f8a-b81c-c6a29dcc7e0a None None] MFA Rules not processed for user `bb44e7aef0c7436d93a7233307b5f095`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 11:06:54.936972 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-5a882641-2fd5-4f8a-b81c-c6a29dcc7e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:06:54.992543 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1462/2935] 10.222.0.15 () {58 vars in 1009 bytes} [Sun Sep 11 11:06:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 11:06:55.037614 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c47bf8-fa3b-404d-a0fd-0f5c2feb646c service neutron] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:55.039059 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-96c47bf8-fa3b-404d-a0fd-0f5c2feb646c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:06:55.118798 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c47bf8-fa3b-404d-a0fd-0f5c2feb646c service neutron] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:55.123755 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb07aa99-e285-4dbc-aafd-0461cc2a1ef6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7ca566e66a45b29e3d02f47bf84981', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:55.124386 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb07aa99-e285-4dbc-aafd-0461cc2a1ef6 service swift] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:55.124662 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb07aa99-e285-4dbc-aafd-0461cc2a1ef6 service swift] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:55.124932 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb07aa99-e285-4dbc-aafd-0461cc2a1ef6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:55.133083 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb07aa99-e285-4dbc-aafd-0461cc2a1ef6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:55.135243 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb07aa99-e285-4dbc-aafd-0461cc2a1ef6 service swift] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:55.143644 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1474/2936] 10.222.0.15 () {60 vars in 1384 bytes} [Sun Sep 11 11:06:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 11:06:55.655844 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd500e-f955-4e44-86ee-dbe5ffbe827a None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:55.656135 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd500e-f955-4e44-86ee-dbe5ffbe827a None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:55.656409 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd500e-f955-4e44-86ee-dbe5ffbe827a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:55.675492 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-9ffd500e-f955-4e44-86ee-dbe5ffbe827a None None] MFA Rules not processed for user `bb44e7aef0c7436d93a7233307b5f095`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 11:06:55.676701 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-9ffd500e-f955-4e44-86ee-dbe5ffbe827a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:06:55.684461 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1463/2937] 10.222.0.15 () {58 vars in 967 bytes} [Sun Sep 11 11:06:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 11:06:55.696898 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb07aa99-e285-4dbc-aafd-0461cc2a1ef6 service swift] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:55.707369 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb07aa99-e285-4dbc-aafd-0461cc2a1ef6 service swift] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:55.712729 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ef1b2f-bf21-4aa9-9df0-cb718ac165d5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7ca566e66a45b29e3d02f47bf84981', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:55.713322 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ef1b2f-bf21-4aa9-9df0-cb718ac165d5 service swift] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:55.713603 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ef1b2f-bf21-4aa9-9df0-cb718ac165d5 service swift] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:55.713876 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ef1b2f-bf21-4aa9-9df0-cb718ac165d5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:55.721750 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ef1b2f-bf21-4aa9-9df0-cb718ac165d5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:55.723322 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ef1b2f-bf21-4aa9-9df0-cb718ac165d5 service swift] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:55.731836 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1475/2938] 10.222.0.15 () {60 vars in 1384 bytes} [Sun Sep 11 11:06:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 11:06:55.954816 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f537e8b-7de0-4e2e-89c6-6e45787a47b0 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:55.955292 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f537e8b-7de0-4e2e-89c6-6e45787a47b0 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:55.955411 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f537e8b-7de0-4e2e-89c6-6e45787a47b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:55.975123 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-3f537e8b-7de0-4e2e-89c6-6e45787a47b0 None None] MFA Rules not processed for user `bb44e7aef0c7436d93a7233307b5f095`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 11:06:55.976356 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-3f537e8b-7de0-4e2e-89c6-6e45787a47b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:06:55.983364 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1464/2939] 10.222.0.15 () {58 vars in 1009 bytes} [Sun Sep 11 11:06:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 11:06:55.995491 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ef1b2f-bf21-4aa9-9df0-cb718ac165d5 service swift] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:56.005910 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ef1b2f-bf21-4aa9-9df0-cb718ac165d5 service swift] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:56.011585 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74d24cb-f086-4ba5-9b0b-e8023051bc08 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7ca566e66a45b29e3d02f47bf84981', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:56.012177 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d24cb-f086-4ba5-9b0b-e8023051bc08 service swift] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:56.012510 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d24cb-f086-4ba5-9b0b-e8023051bc08 service swift] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:56.012730 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d24cb-f086-4ba5-9b0b-e8023051bc08 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:56.030713 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74d24cb-f086-4ba5-9b0b-e8023051bc08 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:56.032344 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74d24cb-f086-4ba5-9b0b-e8023051bc08 service swift] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:56.040801 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1476/2940] 10.222.0.15 () {60 vars in 1384 bytes} [Sun Sep 11 11:06:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 11:06:56.358949 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f170c4c-77d1-4074-9bf5-35f2e537a1e0 None None] REQUEST_METHOD: `POST` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:56.359258 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f170c4c-77d1-4074-9bf5-35f2e537a1e0 None None] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:56.359576 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f170c4c-77d1-4074-9bf5-35f2e537a1e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:56.379337 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.auth.core [None req-9f170c4c-77d1-4074-9bf5-35f2e537a1e0 None None] MFA Rules not processed for user `bb44e7aef0c7436d93a7233307b5f095`. Rule list: `[]` (Enabled: `True`). {{(pid=90653) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 11:06:56.380538 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-9f170c4c-77d1-4074-9bf5-35f2e537a1e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:06:56.388421 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1465/2941] 10.222.0.15 () {58 vars in 967 bytes} [Sun Sep 11 11:06:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 11:06:56.400375 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74d24cb-f086-4ba5-9b0b-e8023051bc08 service swift] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:56.410963 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74d24cb-f086-4ba5-9b0b-e8023051bc08 service swift] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:56.416171 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b914fc2-f3f1-46c6-bbb8-36dc0db44fb9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc7ca566e66a45b29e3d02f47bf84981', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:56.416767 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b914fc2-f3f1-46c6-bbb8-36dc0db44fb9 service swift] REQUEST_METHOD: `GET` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:56.417050 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b914fc2-f3f1-46c6-bbb8-36dc0db44fb9 service swift] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:56.417346 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b914fc2-f3f1-46c6-bbb8-36dc0db44fb9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:56.426341 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b914fc2-f3f1-46c6-bbb8-36dc0db44fb9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:56.427941 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b914fc2-f3f1-46c6-bbb8-36dc0db44fb9 service swift] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:56.436736 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1477/2942] 10.222.0.15 () {60 vars in 1384 bytes} [Sun Sep 11 11:06:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 11:06:58.515591 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f170c4c-77d1-4074-9bf5-35f2e537a1e0 None None] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:58.580709 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f170c4c-77d1-4074-9bf5-35f2e537a1e0 None None] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:58.585245 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8205078-2eea-4b31-8a36-363d06fbc702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:58.585875 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8205078-2eea-4b31-8a36-363d06fbc702 admin admin] REQUEST_METHOD: `DELETE` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:58.586157 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8205078-2eea-4b31-8a36-363d06fbc702 admin admin] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:58.586431 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8205078-2eea-4b31-8a36-363d06fbc702 admin admin] PATH_INFO: `/v3/users/efaa6f24a9f941b0a3dd7d16dd46e56f` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:58.587626 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8205078-2eea-4b31-8a36-363d06fbc702 admin admin] RBAC: Authorizing `identity:delete_user(user_id=efaa6f24a9f941b0a3dd7d16dd46e56f)` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:58.589556 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8205078-2eea-4b31-8a36-363d06fbc702 admin admin] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:58.635651 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-d8205078-2eea-4b31-8a36-363d06fbc702 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efaa6f24a9f941b0a3dd7d16dd46e56f'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:58.642914 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-d8205078-2eea-4b31-8a36-363d06fbc702 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efaa6f24a9f941b0a3dd7d16dd46e56f'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:58.643568 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-d8205078-2eea-4b31-8a36-363d06fbc702 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efaa6f24a9f941b0a3dd7d16dd46e56f'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:58.644175 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-d8205078-2eea-4b31-8a36-363d06fbc702 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efaa6f24a9f941b0a3dd7d16dd46e56f'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:58.651993 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.notifications [None req-d8205078-2eea-4b31-8a36-363d06fbc702 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efaa6f24a9f941b0a3dd7d16dd46e56f'} {{(pid=90653) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:58.662837 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 11:06:58.671257 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1466/2943] 10.222.0.15 () {58 vars in 1224 bytes} [Sun Sep 11 11:06:58 2022] DELETE /identity/v3/users/efaa6f24a9f941b0a3dd7d16dd46e56f => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 11:06:58.675591 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35a22a-28ac-40f6-a590-ea2bf0e22e92 None None] REQUEST_METHOD: `POST` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:58.676100 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35a22a-28ac-40f6-a590-ea2bf0e22e92 None None] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:58.676209 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35a22a-28ac-40f6-a590-ea2bf0e22e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:58.696428 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.auth.core [None req-5e35a22a-28ac-40f6-a590-ea2bf0e22e92 None None] MFA Rules not processed for user `3570ee53f3c442848938999c912a848b`. Rule list: `[]` (Enabled: `True`). {{(pid=90654) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 11:06:58.698024 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-5e35a22a-28ac-40f6-a590-ea2bf0e22e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:06:58.760826 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1478/2944] 10.222.0.15 () {58 vars in 958 bytes} [Sun Sep 11 11:06:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 11:06:58.770440 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8205078-2eea-4b31-8a36-363d06fbc702 admin admin] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:58.771863 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-d8205078-2eea-4b31-8a36-363d06fbc702 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:06:58.829715 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8205078-2eea-4b31-8a36-363d06fbc702 admin admin] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:58.833638 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff27d050-7584-4495-a563-e21f69d97153 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40072bc8dcb5454382a93e20b63f8594', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:58.834235 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff27d050-7584-4495-a563-e21f69d97153 service neutron] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:58.834511 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff27d050-7584-4495-a563-e21f69d97153 service neutron] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:58.834776 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff27d050-7584-4495-a563-e21f69d97153 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:58.844266 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff27d050-7584-4495-a563-e21f69d97153 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:58.845658 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff27d050-7584-4495-a563-e21f69d97153 service neutron] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:58.855805 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1467/2945] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 11:06:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 11:06:59.027317 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e35a22a-28ac-40f6-a590-ea2bf0e22e92 None None] Authenticating user token {{(pid=90654) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:06:59.028751 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.fernet_utils [None req-5e35a22a-28ac-40f6-a590-ea2bf0e22e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90654) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:06:59.043838 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e35a22a-28ac-40f6-a590-ea2bf0e22e92 None None] Validating token access rules against request {{(pid=90654) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:06:59.050686 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0ed8f0-26ad-43b9-8a33-cb9887b21cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3570ee53f3c442848938999c912a848b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dcf7c8cb1c844da09226c671da5a163e', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90654) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:06:59.051332 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0ed8f0-26ad-43b9-8a33-cb9887b21cab admin admin] REQUEST_METHOD: `DELETE` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:06:59.051637 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0ed8f0-26ad-43b9-8a33-cb9887b21cab admin admin] SCRIPT_NAME: `/identity` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:06:59.051912 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0ed8f0-26ad-43b9-8a33-cb9887b21cab admin admin] PATH_INFO: `/v3/projects/2e6c7071912645288d40f666ec2d4582` {{(pid=90654) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:06:59.053737 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0ed8f0-26ad-43b9-8a33-cb9887b21cab admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e6c7071912645288d40f666ec2d4582)` {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:06:59.057708 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0ed8f0-26ad-43b9-8a33-cb9887b21cab admin admin] RBAC: Authorization granted {{(pid=90654) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:06:59.101694 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 11:06:59.112843 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-3f0ed8f0-26ad-43b9-8a33-cb9887b21cab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e6c7071912645288d40f666ec2d4582', 'request_id': 'req-3f0ed8f0-26ad-43b9-8a33-cb9887b21cab'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:59.119633 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-3f0ed8f0-26ad-43b9-8a33-cb9887b21cab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e6c7071912645288d40f666ec2d4582', 'request_id': 'req-3f0ed8f0-26ad-43b9-8a33-cb9887b21cab'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:59.125392 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-3f0ed8f0-26ad-43b9-8a33-cb9887b21cab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e6c7071912645288d40f666ec2d4582', 'request_id': 'req-3f0ed8f0-26ad-43b9-8a33-cb9887b21cab'} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:59.137961 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-3f0ed8f0-26ad-43b9-8a33-cb9887b21cab admin admin] The token cache is being invalidate because project 2e6c7071912645288d40f666ec2d4582 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=90654) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 11:06:59.138355 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-3f0ed8f0-26ad-43b9-8a33-cb9887b21cab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:59.138989 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG keystone.notifications [None req-3f0ed8f0-26ad-43b9-8a33-cb9887b21cab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=90654) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 11:06:59.140484 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: [pid: 90654|app: 0|req: 1479/2946] 10.222.0.15 () {58 vars in 1233 bytes} [Sun Sep 11 11:06:59 2022] DELETE /identity/v3/projects/2e6c7071912645288d40f666ec2d4582 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 11:07:08.850758 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=28 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 11:07:09.134941 dv-cinder-856460-3-iscsi devstack@keystone.service[90654]: DEBUG dbcounter [-] [90654] Writing DB stats keystone:DELETE=5,keystone:SELECT=11,keystone:INSERT=1 {{(pid=90654) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 11:10:01.094194 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff27d050-7584-4495-a563-e21f69d97153 service neutron] Authenticating user token {{(pid=90653) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 11:10:01.095609 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-ff27d050-7584-4495-a563-e21f69d97153 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:10:01.164425 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff27d050-7584-4495-a563-e21f69d97153 service neutron] Validating token access rules against request {{(pid=90653) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 11:10:01.223015 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f59aa26-ef9d-4aa0-abc2-1148fd0a3673 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08477b4c88f04c9e97391aba401879e0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5e1f845f99f04202b83621cb65e0adc2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=90653) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 11:10:01.223650 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f59aa26-ef9d-4aa0-abc2-1148fd0a3673 service placement] REQUEST_METHOD: `GET` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 11:10:01.223937 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f59aa26-ef9d-4aa0-abc2-1148fd0a3673 service placement] SCRIPT_NAME: `/identity` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 11:10:01.224213 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f59aa26-ef9d-4aa0-abc2-1148fd0a3673 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=90653) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 11:10:01.225945 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.fernet_utils [None req-3f59aa26-ef9d-4aa0-abc2-1148fd0a3673 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=90653) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 11:10:01.237003 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f59aa26-ef9d-4aa0-abc2-1148fd0a3673 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 11:10:01.238570 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f59aa26-ef9d-4aa0-abc2-1148fd0a3673 service placement] RBAC: Authorization granted {{(pid=90653) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 11:10:01.248653 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: [pid: 90653|app: 0|req: 1468/2947] 10.222.0.15 () {60 vars in 1365 bytes} [Sun Sep 11 11:10:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 11:10:11.244529 dv-cinder-856460-3-iscsi devstack@keystone.service[90653]: DEBUG dbcounter [-] [90653] Writing DB stats keystone:SELECT=58 {{(pid=90653) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}